Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/cp3d_avg_feats_no_mpnn_from_binary_cross_entropy/config.json --data_path /home/saxelrod/rgb_nfs/dsets/cov_2_cl/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/cp3d_avg_feats_no_mpnn_from_binary_cross_entropy/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/train_full.csv',
 'dataset_type': 'classification',
 'depth': 4,
 'device': device(type='cuda', index=1),
 'dropout': 0.35000000000000003,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': True,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_avg/train_binary_cross_entropy.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 2200,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 2200,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'binary_cross_entropy',
 'metrics': ['binary_cross_entropy'],
 'minimize_score': True,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/cp3d_avg_feats_no_mpnn_from_binary_cross_entropy',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_avg/test_binary_cross_entropy.npz'],
 'separate_test_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_avg/val_binary_cross_entropy.npz'],
 'separate_val_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=300, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.35000000000000003, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 5,506,601
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 33.900494
Epoch 1
Validation binary_cross_entropy = 14.755643
Epoch 2
Validation binary_cross_entropy = 79.202782
Epoch 3
Validation binary_cross_entropy = 8.655228
Epoch 4
Loss = 1.4870e+02, PNorm = 52.2810, GNorm = 2186.3551, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 8.917197
Epoch 5
Validation binary_cross_entropy = 43.301228
Epoch 6
Validation binary_cross_entropy = 21.861624
Epoch 7
Validation binary_cross_entropy = 7.155891
Epoch 8
Validation binary_cross_entropy = 28.350473
Epoch 9
Loss = 7.8725e+01, PNorm = 52.4778, GNorm = 2515.1635, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 38.589146
Epoch 10
Validation binary_cross_entropy = 6.663542
Epoch 11
Validation binary_cross_entropy = 55.676197
Epoch 12
Validation binary_cross_entropy = 21.397789
Epoch 13
Validation binary_cross_entropy = 7.907796
Epoch 14
Loss = 4.3027e+01, PNorm = 52.7092, GNorm = 735.6308, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 15.041706
Epoch 15
Validation binary_cross_entropy = 8.285403
Epoch 16
Validation binary_cross_entropy = 12.752707
Epoch 17
Validation binary_cross_entropy = 11.544869
Epoch 18
Validation binary_cross_entropy = 6.013999
Epoch 19
Loss = 2.8863e+01, PNorm = 52.8816, GNorm = 557.9333, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 8.241272
Epoch 20
Validation binary_cross_entropy = 6.576644
Epoch 21
Validation binary_cross_entropy = 6.527539
Epoch 22
Validation binary_cross_entropy = 13.754533
Epoch 23
Validation binary_cross_entropy = 9.616137
Epoch 24
Loss = 3.7650e+01, PNorm = 53.0103, GNorm = 1110.3742, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 5.013048
Epoch 25
Validation binary_cross_entropy = 5.795916
Epoch 26
Validation binary_cross_entropy = 6.128411
Epoch 27
Validation binary_cross_entropy = 5.995149
Epoch 28
Validation binary_cross_entropy = 14.732641
Epoch 29
Loss = 3.1140e+01, PNorm = 53.1122, GNorm = 781.4689, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 42.539154
Epoch 30
Validation binary_cross_entropy = 5.303289
Epoch 31
Validation binary_cross_entropy = 6.721199
Epoch 32
Validation binary_cross_entropy = 29.944454
Epoch 33
Validation binary_cross_entropy = 45.198296
Epoch 34
Loss = 4.5206e+01, PNorm = 53.2013, GNorm = 681.9078, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 7.160017
Epoch 35
Validation binary_cross_entropy = 6.333936
Epoch 36
Validation binary_cross_entropy = 7.917451
Epoch 37
Validation binary_cross_entropy = 10.300174
Epoch 38
Validation binary_cross_entropy = 16.340590
Epoch 39
Loss = 1.7662e+01, PNorm = 53.2766, GNorm = 414.4870, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 15.879039
Epoch 40
Validation binary_cross_entropy = 22.211998
Epoch 41
Validation binary_cross_entropy = 7.433684
Epoch 42
Validation binary_cross_entropy = 12.917993
Epoch 43
Validation binary_cross_entropy = 6.456704
Epoch 44
Loss = 1.7080e+01, PNorm = 53.3401, GNorm = 469.1711, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 15.705492
Epoch 45
Validation binary_cross_entropy = 13.458341
Epoch 46
Validation binary_cross_entropy = 9.277369
Epoch 47
Validation binary_cross_entropy = 6.894472
Epoch 48
Validation binary_cross_entropy = 12.947133
Epoch 49
Loss = 1.8084e+01, PNorm = 53.3995, GNorm = 349.3979, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 18.811466
Epoch 50
Validation binary_cross_entropy = 11.921856
Epoch 51
Validation binary_cross_entropy = 10.695750
Epoch 52
Validation binary_cross_entropy = 11.991227
Epoch 53
Validation binary_cross_entropy = 12.445008
Epoch 54
Loss = 1.0341e+01, PNorm = 53.4526, GNorm = 292.4166, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 8.769790
Epoch 55
Validation binary_cross_entropy = 7.058466
Epoch 56
Validation binary_cross_entropy = 4.069434
Epoch 57
Validation binary_cross_entropy = 6.954572
Epoch 58
Validation binary_cross_entropy = 3.277186
Epoch 59
Loss = 4.3347e+00, PNorm = 53.4977, GNorm = 359.9431, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 7.850590
Epoch 60
Validation binary_cross_entropy = 7.492659
Epoch 61
Validation binary_cross_entropy = 8.014929
Epoch 62
Validation binary_cross_entropy = 7.943722
Epoch 63
Validation binary_cross_entropy = 6.827281
Epoch 64
Loss = 8.4897e+00, PNorm = 53.5385, GNorm = 407.2310, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 13.139604
Epoch 65
Validation binary_cross_entropy = 8.055348
Epoch 66
Validation binary_cross_entropy = 4.969389
Epoch 67
Validation binary_cross_entropy = 3.761002
Epoch 68
Validation binary_cross_entropy = 3.653679
Epoch 69
Loss = 9.1998e+00, PNorm = 53.5757, GNorm = 151.7243, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 7.881752
Epoch 70
Validation binary_cross_entropy = 8.842993
Epoch 71
Validation binary_cross_entropy = 3.434149
Epoch 72
Validation binary_cross_entropy = 1.885134
Epoch 73
Validation binary_cross_entropy = 1.571434
Epoch 74
Loss = 7.4003e+00, PNorm = 53.6115, GNorm = 261.6093, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 4.561809
Epoch 75
Validation binary_cross_entropy = 6.061508
Epoch 76
Validation binary_cross_entropy = 2.394319
Epoch 77
Validation binary_cross_entropy = 1.653959
Epoch 78
Validation binary_cross_entropy = 2.148764
Epoch 79
Loss = 3.3858e+00, PNorm = 53.6434, GNorm = 158.2334, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 3.700638
Epoch 80
Validation binary_cross_entropy = 6.985490
Epoch 81
Validation binary_cross_entropy = 4.898744
Epoch 82
Validation binary_cross_entropy = 3.077458
Epoch 83
Validation binary_cross_entropy = 1.834308
Epoch 84
Loss = 4.1742e+00, PNorm = 53.6709, GNorm = 140.4244, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 1.302520
Epoch 85
Validation binary_cross_entropy = 1.243966
Epoch 86
Validation binary_cross_entropy = 1.577413
Epoch 87
Validation binary_cross_entropy = 1.717795
Epoch 88
Validation binary_cross_entropy = 1.985128
Epoch 89
Loss = 2.3206e+00, PNorm = 53.6938, GNorm = 163.1693, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 3.427387
Epoch 90
Validation binary_cross_entropy = 1.733326
Epoch 91
Validation binary_cross_entropy = 1.343381
Epoch 92
Validation binary_cross_entropy = 0.988158
Epoch 93
Validation binary_cross_entropy = 0.872371
Epoch 94
Loss = 2.8137e+00, PNorm = 53.7154, GNorm = 163.0662, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.944071
Epoch 95
Validation binary_cross_entropy = 1.680136
Epoch 96
Validation binary_cross_entropy = 1.641104
Epoch 97
Validation binary_cross_entropy = 1.118174
Epoch 98
Validation binary_cross_entropy = 1.085752
Epoch 99
Loss = 1.8751e+00, PNorm = 53.7395, GNorm = 119.9967, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 1.077387
Epoch 100
Validation binary_cross_entropy = 1.021748
Epoch 101
Validation binary_cross_entropy = 0.846419
Epoch 102
Validation binary_cross_entropy = 1.456513
Epoch 103
Validation binary_cross_entropy = 2.475103
Epoch 104
Loss = 2.5141e+00, PNorm = 53.7624, GNorm = 191.5542, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 1.299044
Epoch 105
Validation binary_cross_entropy = 0.761991
Epoch 106
Validation binary_cross_entropy = 0.836744
Epoch 107
Validation binary_cross_entropy = 0.638251
Epoch 108
Validation binary_cross_entropy = 0.628363
Epoch 109
Loss = 3.8620e+00, PNorm = 53.7835, GNorm = 233.5213, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.862680
Epoch 110
Validation binary_cross_entropy = 0.752447
Epoch 111
Validation binary_cross_entropy = 1.682902
Epoch 112
Validation binary_cross_entropy = 2.354743
Epoch 113
Validation binary_cross_entropy = 1.109453
Epoch 114
Loss = 2.0573e+00, PNorm = 53.8048, GNorm = 112.3151, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.718774
Epoch 115
Validation binary_cross_entropy = 0.691165
Epoch 116
Validation binary_cross_entropy = 0.672616
Epoch 117
Validation binary_cross_entropy = 0.666817
Epoch 118
Validation binary_cross_entropy = 0.648698
Epoch 119
Loss = 1.2995e+00, PNorm = 53.8229, GNorm = 68.9022, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.627949
Epoch 120
Validation binary_cross_entropy = 0.643989
Epoch 121
Validation binary_cross_entropy = 0.633320
Epoch 122
Validation binary_cross_entropy = 0.936506
Epoch 123
Validation binary_cross_entropy = 1.326450
Epoch 124
Loss = 3.4461e+00, PNorm = 53.8388, GNorm = 388.4899, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.799615
Epoch 125
Validation binary_cross_entropy = 0.696509
Epoch 126
Validation binary_cross_entropy = 0.683082
Epoch 127
Validation binary_cross_entropy = 0.756540
Epoch 128
Validation binary_cross_entropy = 0.895027
Epoch 129
Loss = 1.5569e+00, PNorm = 53.8574, GNorm = 90.8721, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 1.002792
Epoch 130
Validation binary_cross_entropy = 1.077702
Epoch 131
Validation binary_cross_entropy = 1.081589
Epoch 132
Validation binary_cross_entropy = 0.956647
Epoch 133
Validation binary_cross_entropy = 0.700834
Epoch 134
Loss = 1.2965e+00, PNorm = 53.8734, GNorm = 48.7331, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.640861
Epoch 135
Validation binary_cross_entropy = 0.728759
Epoch 136
Validation binary_cross_entropy = 1.033207
Epoch 137
Validation binary_cross_entropy = 1.409574
Epoch 138
Validation binary_cross_entropy = 1.282658
Epoch 139
Loss = 1.2588e+00, PNorm = 53.8884, GNorm = 98.6971, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.734142
Epoch 140
Validation binary_cross_entropy = 0.495765
Epoch 141
Validation binary_cross_entropy = 0.471128
Epoch 142
Validation binary_cross_entropy = 0.571533
Epoch 143
Validation binary_cross_entropy = 1.001230
Epoch 144
Loss = 1.4503e+00, PNorm = 53.9012, GNorm = 74.8317, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 1.047386
Epoch 145
Validation binary_cross_entropy = 0.891596
Epoch 146
Validation binary_cross_entropy = 0.778522
Epoch 147
Validation binary_cross_entropy = 0.752954
Epoch 148
Validation binary_cross_entropy = 0.663424
Epoch 149
Loss = 9.1333e-01, PNorm = 53.9125, GNorm = 67.3025, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.632406
Epoch 150
Validation binary_cross_entropy = 0.657631
Epoch 151
Validation binary_cross_entropy = 0.626725
Epoch 152
Validation binary_cross_entropy = 0.527681
Epoch 153
Validation binary_cross_entropy = 0.459873
Epoch 154
Loss = 1.2768e+00, PNorm = 53.9226, GNorm = 78.4708, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.404704
Epoch 155
Validation binary_cross_entropy = 0.398173
Epoch 156
Validation binary_cross_entropy = 0.416939
Epoch 157
Validation binary_cross_entropy = 0.625268
Epoch 158
Validation binary_cross_entropy = 0.687100
Epoch 159
Loss = 1.2860e+00, PNorm = 53.9322, GNorm = 76.9322, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.592368
Epoch 160
Validation binary_cross_entropy = 0.659709
Epoch 161
Validation binary_cross_entropy = 0.675530
Epoch 162
Validation binary_cross_entropy = 0.653448
Epoch 163
Validation binary_cross_entropy = 0.594768
Epoch 164
Loss = 1.1169e+00, PNorm = 53.9411, GNorm = 105.9186, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.599997
Epoch 165
Validation binary_cross_entropy = 0.566163
Epoch 166
Validation binary_cross_entropy = 0.492261
Epoch 167
Validation binary_cross_entropy = 0.471767
Epoch 168
Validation binary_cross_entropy = 0.376399
Epoch 169
Loss = 7.8825e-01, PNorm = 53.9500, GNorm = 76.8882, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.377967
Epoch 170
Validation binary_cross_entropy = 0.391077
Epoch 171
Validation binary_cross_entropy = 0.385476
Epoch 172
Validation binary_cross_entropy = 0.437792
Epoch 173
Validation binary_cross_entropy = 0.608577
Epoch 174
Loss = 1.2237e+00, PNorm = 53.9587, GNorm = 62.9233, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.647030
Epoch 175
Validation binary_cross_entropy = 0.589194
Epoch 176
Validation binary_cross_entropy = 0.542249
Epoch 177
Validation binary_cross_entropy = 0.543761
Epoch 178
Validation binary_cross_entropy = 0.565652
Epoch 179
Loss = 1.2338e+00, PNorm = 53.9662, GNorm = 93.1934, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.639998
Epoch 180
Validation binary_cross_entropy = 0.835901
Epoch 181
Validation binary_cross_entropy = 0.805764
Epoch 182
Validation binary_cross_entropy = 0.519906
Epoch 183
Validation binary_cross_entropy = 0.473712
Epoch 184
Loss = 6.2033e-01, PNorm = 53.9736, GNorm = 59.8511, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.540002
Epoch 185
Validation binary_cross_entropy = 0.575958
Epoch 186
Validation binary_cross_entropy = 0.450193
Epoch 187
Validation binary_cross_entropy = 0.500956
Epoch 188
Validation binary_cross_entropy = 0.658501
Epoch 189
Loss = 1.0584e+00, PNorm = 53.9817, GNorm = 91.3705, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.934360
Epoch 190
Validation binary_cross_entropy = 1.069394
Epoch 191
Validation binary_cross_entropy = 0.683846
Epoch 192
Validation binary_cross_entropy = 0.489635
Epoch 193
Validation binary_cross_entropy = 0.497932
Epoch 194
Loss = 1.2811e+00, PNorm = 53.9893, GNorm = 94.3936, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.537605
Epoch 195
Validation binary_cross_entropy = 0.673322
Epoch 196
Validation binary_cross_entropy = 1.003397
Epoch 197
Validation binary_cross_entropy = 1.138387
Epoch 198
Validation binary_cross_entropy = 1.000773
Epoch 199
Loss = 9.3757e-01, PNorm = 53.9973, GNorm = 104.9999, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.694850
Epoch 200
Validation binary_cross_entropy = 0.537744
Epoch 201
Validation binary_cross_entropy = 0.485502
Epoch 202
Validation binary_cross_entropy = 0.483236
Epoch 203
Validation binary_cross_entropy = 0.444137
Epoch 204
Loss = 8.1821e-01, PNorm = 54.0050, GNorm = 54.3789, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.450732
Epoch 205
Validation binary_cross_entropy = 0.459099
Epoch 206
Validation binary_cross_entropy = 0.456000
Epoch 207
Validation binary_cross_entropy = 0.456679
Epoch 208
Validation binary_cross_entropy = 0.449562
Epoch 209
Loss = 9.4858e-01, PNorm = 54.0116, GNorm = 71.6160, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.437931
Epoch 210
Validation binary_cross_entropy = 0.430077
Epoch 211
Validation binary_cross_entropy = 0.424642
Epoch 212
Validation binary_cross_entropy = 0.418753
Epoch 213
Validation binary_cross_entropy = 0.415579
Epoch 214
Loss = 6.5466e-01, PNorm = 54.0175, GNorm = 90.9867, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.435716
Epoch 215
Validation binary_cross_entropy = 0.469367
Epoch 216
Validation binary_cross_entropy = 0.506047
Epoch 217
Validation binary_cross_entropy = 0.519505
Epoch 218
Validation binary_cross_entropy = 0.517376
Epoch 219
Loss = 7.7291e-01, PNorm = 54.0235, GNorm = 55.7603, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.499678
Epoch 220
Validation binary_cross_entropy = 0.485104
Epoch 221
Validation binary_cross_entropy = 0.468661
Epoch 222
Validation binary_cross_entropy = 0.450138
Epoch 223
Validation binary_cross_entropy = 0.440532
Epoch 224
Loss = 1.0003e+00, PNorm = 54.0285, GNorm = 105.5033, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.435416
Epoch 225
Validation binary_cross_entropy = 0.430963
Epoch 226
Validation binary_cross_entropy = 0.437791
Epoch 227
Validation binary_cross_entropy = 0.458504
Epoch 228
Validation binary_cross_entropy = 0.491008
Epoch 229
Loss = 4.9838e-01, PNorm = 54.0328, GNorm = 46.0294, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.521496
Epoch 230
Validation binary_cross_entropy = 0.555427
Epoch 231
Validation binary_cross_entropy = 0.568724
Epoch 232
Validation binary_cross_entropy = 0.577833
Epoch 233
Validation binary_cross_entropy = 0.580549
Epoch 234
Loss = 5.2554e-01, PNorm = 54.0369, GNorm = 55.5156, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.564093
Epoch 235
Validation binary_cross_entropy = 0.542185
Epoch 236
Validation binary_cross_entropy = 0.516462
Epoch 237
Validation binary_cross_entropy = 0.488350
Epoch 238
Validation binary_cross_entropy = 0.482863
Epoch 239
Loss = 5.7269e-01, PNorm = 54.0414, GNorm = 28.4280, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.490484
Epoch 240
Validation binary_cross_entropy = 0.509291
Epoch 241
Validation binary_cross_entropy = 0.534220
Epoch 242
Validation binary_cross_entropy = 0.548747
Epoch 243
Validation binary_cross_entropy = 0.525524
Epoch 244
Loss = 4.9420e-01, PNorm = 54.0465, GNorm = 44.6628, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.493092
Epoch 245
Validation binary_cross_entropy = 0.453853
Epoch 246
Validation binary_cross_entropy = 0.419563
Epoch 247
Validation binary_cross_entropy = 0.409577
Epoch 248
Validation binary_cross_entropy = 0.423127
Epoch 249
Loss = 6.7837e-01, PNorm = 54.0512, GNorm = 59.6663, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.450132
Epoch 250
Validation binary_cross_entropy = 0.484888
Epoch 251
Validation binary_cross_entropy = 0.514811
Epoch 252
Validation binary_cross_entropy = 0.536251
Epoch 253
Validation binary_cross_entropy = 0.548688
Epoch 254
Loss = 5.9031e-01, PNorm = 54.0556, GNorm = 35.5585, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.562810
Epoch 255
Validation binary_cross_entropy = 0.569939
Epoch 256
Validation binary_cross_entropy = 0.545707
Epoch 257
Validation binary_cross_entropy = 0.518487
Epoch 258
Validation binary_cross_entropy = 0.472117
Epoch 259
Loss = 6.7759e-01, PNorm = 54.0605, GNorm = 65.4692, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.458453
Epoch 260
Validation binary_cross_entropy = 0.441263
Epoch 261
Validation binary_cross_entropy = 0.449406
Epoch 262
Validation binary_cross_entropy = 0.494943
Epoch 263
Validation binary_cross_entropy = 0.529601
Epoch 264
Loss = 8.9468e-01, PNorm = 54.0659, GNorm = 71.0078, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.523646
Epoch 265
Validation binary_cross_entropy = 0.495497
Epoch 266
Validation binary_cross_entropy = 0.476253
Epoch 267
Validation binary_cross_entropy = 0.481158
Epoch 268
Validation binary_cross_entropy = 0.492479
Epoch 269
Loss = 5.2424e-01, PNorm = 54.0709, GNorm = 46.9348, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.508609
Epoch 270
Validation binary_cross_entropy = 0.521235
Epoch 271
Validation binary_cross_entropy = 0.526007
Epoch 272
Validation binary_cross_entropy = 0.517842
Epoch 273
Validation binary_cross_entropy = 0.504651
Epoch 274
Loss = 5.4512e-01, PNorm = 54.0760, GNorm = 62.6490, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.501599
Epoch 275
Validation binary_cross_entropy = 0.506654
Epoch 276
Validation binary_cross_entropy = 0.506997
Epoch 277
Validation binary_cross_entropy = 0.499128
Epoch 278
Validation binary_cross_entropy = 0.482226
Epoch 279
Loss = 1.0044e+00, PNorm = 54.0804, GNorm = 69.6544, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.475293
Epoch 280
Validation binary_cross_entropy = 0.488954
Epoch 281
Validation binary_cross_entropy = 0.511014
Epoch 282
Validation binary_cross_entropy = 0.536731
Epoch 283
Validation binary_cross_entropy = 0.547443
Epoch 284
Loss = 6.5718e-01, PNorm = 54.0846, GNorm = 43.7971, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.514739
Epoch 285
Validation binary_cross_entropy = 0.494145
Epoch 286
Validation binary_cross_entropy = 0.461726
Epoch 287
Validation binary_cross_entropy = 0.428435
Epoch 288
Validation binary_cross_entropy = 0.410309
Epoch 289
Loss = 4.4702e-01, PNorm = 54.0887, GNorm = 39.0563, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.411545
Epoch 290
Validation binary_cross_entropy = 0.437701
Epoch 291
Validation binary_cross_entropy = 0.468654
Epoch 292
Validation binary_cross_entropy = 0.494260
Epoch 293
Validation binary_cross_entropy = 0.509055
Epoch 294
Loss = 7.0068e-01, PNorm = 54.0931, GNorm = 57.3247, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.507211
Epoch 295
Validation binary_cross_entropy = 0.502775
Epoch 296
Validation binary_cross_entropy = 0.491939
Epoch 297
Validation binary_cross_entropy = 0.479707
Epoch 298
Validation binary_cross_entropy = 0.476707
Epoch 299
Loss = 7.0745e-01, PNorm = 54.0974, GNorm = 46.2039, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.478215
Model 0 best validation binary_cross_entropy = 0.376399 on epoch 168
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.588609
Ensemble test binary_cross_entropy = 0.588609
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=300, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.35000000000000003, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 5,506,601
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 48.824955
Epoch 1
Validation binary_cross_entropy = 9.667910
Epoch 2
Validation binary_cross_entropy = 57.416733
Epoch 3
Validation binary_cross_entropy = 8.235247
Epoch 4
Loss = 1.0925e+02, PNorm = 52.2784, GNorm = 2454.1352, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 13.415585
Epoch 5
Validation binary_cross_entropy = 62.768139
Epoch 6
Validation binary_cross_entropy = 7.497529
Epoch 7
Validation binary_cross_entropy = 11.323205
Epoch 8
Validation binary_cross_entropy = 72.326691
Epoch 9
Loss = 6.4194e+01, PNorm = 52.4723, GNorm = 910.1951, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 8.369984
Epoch 10
Validation binary_cross_entropy = 10.838732
Epoch 11
Validation binary_cross_entropy = 21.519855
Epoch 12
Validation binary_cross_entropy = 9.511091
Epoch 13
Validation binary_cross_entropy = 31.330418
Epoch 14
Loss = 7.1579e+01, PNorm = 52.6998, GNorm = 1501.2338, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 13.212402
Epoch 15
Validation binary_cross_entropy = 8.610887
Epoch 16
Validation binary_cross_entropy = 29.199804
Epoch 17
Validation binary_cross_entropy = 39.314312
Epoch 18
Validation binary_cross_entropy = 6.171131
Epoch 19
Loss = 4.0937e+01, PNorm = 52.8787, GNorm = 925.4420, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 30.764845
Epoch 20
Validation binary_cross_entropy = 71.343819
Epoch 21
Validation binary_cross_entropy = 9.867164
Epoch 22
Validation binary_cross_entropy = 15.800875
Epoch 23
Validation binary_cross_entropy = 28.049681
Epoch 24
Loss = 3.5325e+01, PNorm = 53.0160, GNorm = 1092.4619, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 7.227002
Epoch 25
Validation binary_cross_entropy = 6.369212
Epoch 26
Validation binary_cross_entropy = 15.937721
Epoch 27
Validation binary_cross_entropy = 6.285030
Epoch 28
Validation binary_cross_entropy = 5.430871
Epoch 29
Loss = 2.5841e+01, PNorm = 53.1272, GNorm = 797.0525, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 13.548246
Epoch 30
Validation binary_cross_entropy = 37.592869
Epoch 31
Validation binary_cross_entropy = 7.379001
Epoch 32
Validation binary_cross_entropy = 7.589833
Epoch 33
Validation binary_cross_entropy = 9.221426
Epoch 34
Loss = 1.9946e+01, PNorm = 53.2149, GNorm = 534.6459, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 49.290970
Epoch 35
Validation binary_cross_entropy = 12.564527
Epoch 36
Validation binary_cross_entropy = 6.208930
Epoch 37
Validation binary_cross_entropy = 5.484391
Epoch 38
Validation binary_cross_entropy = 7.066130
Epoch 39
Loss = 1.4562e+01, PNorm = 53.2857, GNorm = 268.5458, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 16.943150
Epoch 40
Validation binary_cross_entropy = 16.390982
Epoch 41
Validation binary_cross_entropy = 5.123115
Epoch 42
Validation binary_cross_entropy = 4.852419
Epoch 43
Validation binary_cross_entropy = 6.600029
Epoch 44
Loss = 1.1156e+01, PNorm = 53.3406, GNorm = 275.3622, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 25.331284
Epoch 45
Validation binary_cross_entropy = 24.437878
Epoch 46
Validation binary_cross_entropy = 4.785893
Epoch 47
Validation binary_cross_entropy = 3.789532
Epoch 48
Validation binary_cross_entropy = 3.548692
Epoch 49
Loss = 9.5627e+00, PNorm = 53.3874, GNorm = 282.4875, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 7.145744
Epoch 50
Validation binary_cross_entropy = 19.466351
Epoch 51
Validation binary_cross_entropy = 9.876974
Epoch 52
Validation binary_cross_entropy = 3.414056
Epoch 53
Validation binary_cross_entropy = 3.568633
Epoch 54
Loss = 1.0976e+01, PNorm = 53.4268, GNorm = 239.9572, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 2.731334
Epoch 55
Validation binary_cross_entropy = 8.510286
Epoch 56
Validation binary_cross_entropy = 43.031425
Epoch 57
Validation binary_cross_entropy = 21.378923
Epoch 58
Validation binary_cross_entropy = 2.939581
Epoch 59
Loss = 3.5387e+00, PNorm = 53.4614, GNorm = 155.7207, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 1.892730
Epoch 60
Validation binary_cross_entropy = 2.024862
Epoch 61
Validation binary_cross_entropy = 2.119962
Epoch 62
Validation binary_cross_entropy = 9.786282
Epoch 63
Validation binary_cross_entropy = 17.836227
Epoch 64
Loss = 9.3638e+00, PNorm = 53.4953, GNorm = 275.0972, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 4.100079
Epoch 65
Validation binary_cross_entropy = 1.920056
Epoch 66
Validation binary_cross_entropy = 1.236556
Epoch 67
Validation binary_cross_entropy = 1.316485
Epoch 68
Validation binary_cross_entropy = 2.095488
Epoch 69
Loss = 4.3824e+00, PNorm = 53.5264, GNorm = 189.7710, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 3.532137
Epoch 70
Validation binary_cross_entropy = 2.947313
Epoch 71
Validation binary_cross_entropy = 1.785624
Epoch 72
Validation binary_cross_entropy = 1.725936
Epoch 73
Validation binary_cross_entropy = 0.920653
Epoch 74
Loss = 5.2828e+00, PNorm = 53.5535, GNorm = 209.9848, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.847637
Epoch 75
Validation binary_cross_entropy = 1.298001
Epoch 76
Validation binary_cross_entropy = 2.167429
Epoch 77
Validation binary_cross_entropy = 2.194237
Epoch 78
Validation binary_cross_entropy = 1.934621
Epoch 79
Loss = 2.4122e+00, PNorm = 53.5773, GNorm = 201.7992, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 1.620417
Epoch 80
Validation binary_cross_entropy = 1.854781
Epoch 81
Validation binary_cross_entropy = 3.718171
Epoch 82
Validation binary_cross_entropy = 4.419476
Epoch 83
Validation binary_cross_entropy = 2.259257
Epoch 84
Loss = 1.4691e+00, PNorm = 53.5980, GNorm = 79.6270, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 1.329993
Epoch 85
Validation binary_cross_entropy = 0.700331
Epoch 86
Validation binary_cross_entropy = 0.804303
Epoch 87
Validation binary_cross_entropy = 1.616176
Epoch 88
Validation binary_cross_entropy = 0.863329
Epoch 89
Loss = 2.6324e+00, PNorm = 53.6172, GNorm = 117.9443, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.686726
Epoch 90
Validation binary_cross_entropy = 0.627014
Epoch 91
Validation binary_cross_entropy = 0.630451
Epoch 92
Validation binary_cross_entropy = 0.758095
Epoch 93
Validation binary_cross_entropy = 1.201734
Epoch 94
Loss = 2.6544e+00, PNorm = 53.6355, GNorm = 183.7744, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 1.097060
Epoch 95
Validation binary_cross_entropy = 0.475415
Epoch 96
Validation binary_cross_entropy = 0.466178
Epoch 97
Validation binary_cross_entropy = 0.649764
Epoch 98
Validation binary_cross_entropy = 1.021713
Epoch 99
Loss = 1.6906e+00, PNorm = 53.6554, GNorm = 95.6458, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 2.073131
Epoch 100
Validation binary_cross_entropy = 1.604332
Epoch 101
Validation binary_cross_entropy = 0.792656
Epoch 102
Validation binary_cross_entropy = 0.531058
Epoch 103
Validation binary_cross_entropy = 0.506547
Epoch 104
Loss = 1.0145e+00, PNorm = 53.6745, GNorm = 65.9965, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.499245
Epoch 105
Validation binary_cross_entropy = 0.600768
Epoch 106
Validation binary_cross_entropy = 1.185409
Epoch 107
Validation binary_cross_entropy = 1.410235
Epoch 108
Validation binary_cross_entropy = 0.980121
Epoch 109
Loss = 1.7094e+00, PNorm = 53.6923, GNorm = 119.9269, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.770546
Epoch 110
Validation binary_cross_entropy = 0.668771
Epoch 111
Validation binary_cross_entropy = 0.933950
Epoch 112
Validation binary_cross_entropy = 1.088344
Epoch 113
Validation binary_cross_entropy = 1.436421
Epoch 114
Loss = 1.2687e+00, PNorm = 53.7093, GNorm = 130.5156, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 1.200237
Epoch 115
Validation binary_cross_entropy = 0.633127
Epoch 116
Validation binary_cross_entropy = 0.428209
Epoch 117
Validation binary_cross_entropy = 0.331497
Epoch 118
Validation binary_cross_entropy = 0.349548
Epoch 119
Loss = 2.7566e+00, PNorm = 53.7245, GNorm = 179.1298, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.495992
Epoch 120
Validation binary_cross_entropy = 0.670065
Epoch 121
Validation binary_cross_entropy = 0.807728
Epoch 122
Validation binary_cross_entropy = 0.756709
Epoch 123
Validation binary_cross_entropy = 0.597671
Epoch 124
Loss = 2.4847e+00, PNorm = 53.7398, GNorm = 133.0614, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.462039
Epoch 125
Validation binary_cross_entropy = 0.358957
Epoch 126
Validation binary_cross_entropy = 0.340290
Epoch 127
Validation binary_cross_entropy = 0.377185
Epoch 128
Validation binary_cross_entropy = 0.357957
Epoch 129
Loss = 2.9601e+00, PNorm = 53.7550, GNorm = 273.0114, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.465459
Epoch 130
Validation binary_cross_entropy = 0.453676
Epoch 131
Validation binary_cross_entropy = 0.529012
Epoch 132
Validation binary_cross_entropy = 0.870709
Epoch 133
Validation binary_cross_entropy = 0.734995
Epoch 134
Loss = 1.5203e+00, PNorm = 53.7714, GNorm = 74.5137, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.616508
Epoch 135
Validation binary_cross_entropy = 0.519058
Epoch 136
Validation binary_cross_entropy = 0.479595
Epoch 137
Validation binary_cross_entropy = 0.471100
Epoch 138
Validation binary_cross_entropy = 0.518432
Epoch 139
Loss = 1.0498e+00, PNorm = 53.7862, GNorm = 107.8585, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.586967
Epoch 140
Validation binary_cross_entropy = 0.621842
Epoch 141
Validation binary_cross_entropy = 0.541252
Epoch 142
Validation binary_cross_entropy = 0.464937
Epoch 143
Validation binary_cross_entropy = 0.465520
Epoch 144
Loss = 1.3301e+00, PNorm = 53.7981, GNorm = 114.6001, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.502569
Epoch 145
Validation binary_cross_entropy = 0.643740
Epoch 146
Validation binary_cross_entropy = 0.982705
Epoch 147
Validation binary_cross_entropy = 1.266035
Epoch 148
Validation binary_cross_entropy = 1.016443
Epoch 149
Loss = 1.0768e+00, PNorm = 53.8100, GNorm = 105.0205, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.631456
Epoch 150
Validation binary_cross_entropy = 0.493604
Epoch 151
Validation binary_cross_entropy = 0.505794
Epoch 152
Validation binary_cross_entropy = 0.555687
Epoch 153
Validation binary_cross_entropy = 0.623004
Epoch 154
Loss = 1.1109e+00, PNorm = 53.8226, GNorm = 197.7159, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.686461
Epoch 155
Validation binary_cross_entropy = 0.665605
Epoch 156
Validation binary_cross_entropy = 0.585239
Epoch 157
Validation binary_cross_entropy = 0.519281
Epoch 158
Validation binary_cross_entropy = 0.486876
Epoch 159
Loss = 8.2129e-01, PNorm = 53.8342, GNorm = 92.5361, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.526087
Epoch 160
Validation binary_cross_entropy = 0.743992
Epoch 161
Validation binary_cross_entropy = 0.851872
Epoch 162
Validation binary_cross_entropy = 0.624697
Epoch 163
Validation binary_cross_entropy = 0.542555
Epoch 164
Loss = 8.9836e-01, PNorm = 53.8443, GNorm = 59.6304, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.514764
Epoch 165
Validation binary_cross_entropy = 0.547037
Epoch 166
Validation binary_cross_entropy = 0.603598
Epoch 167
Validation binary_cross_entropy = 0.655519
Epoch 168
Validation binary_cross_entropy = 0.723221
Epoch 169
Loss = 7.2513e-01, PNorm = 53.8538, GNorm = 76.7246, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.691305
Epoch 170
Validation binary_cross_entropy = 0.556669
Epoch 171
Validation binary_cross_entropy = 0.474309
Epoch 172
Validation binary_cross_entropy = 0.428386
Epoch 173
Validation binary_cross_entropy = 0.436606
Epoch 174
Loss = 1.2361e+00, PNorm = 53.8634, GNorm = 118.2002, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.469709
Epoch 175
Validation binary_cross_entropy = 0.502511
Epoch 176
Validation binary_cross_entropy = 0.485800
Epoch 177
Validation binary_cross_entropy = 0.500640
Epoch 178
Validation binary_cross_entropy = 0.520049
Epoch 179
Loss = 1.4141e+00, PNorm = 53.8735, GNorm = 83.1539, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.530929
Epoch 180
Validation binary_cross_entropy = 0.477204
Epoch 181
Validation binary_cross_entropy = 0.521618
Epoch 182
Validation binary_cross_entropy = 0.704612
Epoch 183
Validation binary_cross_entropy = 0.822365
Epoch 184
Loss = 8.5312e-01, PNorm = 53.8831, GNorm = 66.0457, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.793895
Epoch 185
Validation binary_cross_entropy = 0.717607
Epoch 186
Validation binary_cross_entropy = 0.604498
Epoch 187
Validation binary_cross_entropy = 0.502093
Epoch 188
Validation binary_cross_entropy = 0.444169
Epoch 189
Loss = 8.1665e-01, PNorm = 53.8920, GNorm = 46.5507, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.464913
Epoch 190
Validation binary_cross_entropy = 0.477176
Epoch 191
Validation binary_cross_entropy = 0.501124
Epoch 192
Validation binary_cross_entropy = 0.508737
Epoch 193
Validation binary_cross_entropy = 0.520291
Epoch 194
Loss = 8.7116e-01, PNorm = 53.8999, GNorm = 40.7074, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.565732
Epoch 195
Validation binary_cross_entropy = 0.532896
Epoch 196
Validation binary_cross_entropy = 0.501629
Epoch 197
Validation binary_cross_entropy = 0.493395
Epoch 198
Validation binary_cross_entropy = 0.464496
Epoch 199
Loss = 9.6189e-01, PNorm = 53.9082, GNorm = 77.5111, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.467296
Epoch 200
Validation binary_cross_entropy = 0.505668
Epoch 201
Validation binary_cross_entropy = 0.487024
Epoch 202
Validation binary_cross_entropy = 0.486367
Epoch 203
Validation binary_cross_entropy = 0.507878
Epoch 204
Loss = 9.3721e-01, PNorm = 53.9170, GNorm = 70.4215, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.525839
Epoch 205
Validation binary_cross_entropy = 0.572908
Epoch 206
Validation binary_cross_entropy = 0.654089
Epoch 207
Validation binary_cross_entropy = 0.793607
Epoch 208
Validation binary_cross_entropy = 0.832357
Epoch 209
Loss = 9.2364e-01, PNorm = 53.9261, GNorm = 89.0193, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.741605
Epoch 210
Validation binary_cross_entropy = 0.761015
Epoch 211
Validation binary_cross_entropy = 0.762825
Epoch 212
Validation binary_cross_entropy = 0.641924
Epoch 213
Validation binary_cross_entropy = 0.521255
Epoch 214
Loss = 6.0689e-01, PNorm = 53.9346, GNorm = 47.9522, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.441731
Epoch 215
Validation binary_cross_entropy = 0.417412
Epoch 216
Validation binary_cross_entropy = 0.418443
Epoch 217
Validation binary_cross_entropy = 0.428719
Epoch 218
Validation binary_cross_entropy = 0.420555
Epoch 219
Loss = 6.6670e-01, PNorm = 53.9415, GNorm = 44.0788, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.385882
Epoch 220
Validation binary_cross_entropy = 0.381887
Epoch 221
Validation binary_cross_entropy = 0.371334
Epoch 222
Validation binary_cross_entropy = 0.375315
Epoch 223
Validation binary_cross_entropy = 0.385440
Epoch 224
Loss = 7.6556e-01, PNorm = 53.9471, GNorm = 62.8593, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.403540
Epoch 225
Validation binary_cross_entropy = 0.438354
Epoch 226
Validation binary_cross_entropy = 0.497854
Epoch 227
Validation binary_cross_entropy = 0.538051
Epoch 228
Validation binary_cross_entropy = 0.526145
Epoch 229
Loss = 5.0915e-01, PNorm = 53.9528, GNorm = 42.2123, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.524658
Epoch 230
Validation binary_cross_entropy = 0.577287
Epoch 231
Validation binary_cross_entropy = 0.656890
Epoch 232
Validation binary_cross_entropy = 0.719379
Epoch 233
Validation binary_cross_entropy = 0.662598
Epoch 234
Loss = 5.7880e-01, PNorm = 53.9577, GNorm = 43.8373, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.513661
Epoch 235
Validation binary_cross_entropy = 0.449164
Epoch 236
Validation binary_cross_entropy = 0.443232
Epoch 237
Validation binary_cross_entropy = 0.497973
Epoch 238
Validation binary_cross_entropy = 0.533159
Epoch 239
Loss = 8.2067e-01, PNorm = 53.9624, GNorm = 61.3435, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.463413
Epoch 240
Validation binary_cross_entropy = 0.438638
Epoch 241
Validation binary_cross_entropy = 0.442295
Epoch 242
Validation binary_cross_entropy = 0.464314
Epoch 243
Validation binary_cross_entropy = 0.471416
Epoch 244
Loss = 8.1258e-01, PNorm = 53.9675, GNorm = 48.9171, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.474231
Epoch 245
Validation binary_cross_entropy = 0.447581
Epoch 246
Validation binary_cross_entropy = 0.447295
Epoch 247
Validation binary_cross_entropy = 0.463569
Epoch 248
Validation binary_cross_entropy = 0.492313
Epoch 249
Loss = 5.1769e-01, PNorm = 53.9734, GNorm = 48.5451, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.532607
Epoch 250
Validation binary_cross_entropy = 0.553140
Epoch 251
Validation binary_cross_entropy = 0.558616
Epoch 252
Validation binary_cross_entropy = 0.529284
Epoch 253
Validation binary_cross_entropy = 0.517937
Epoch 254
Loss = 9.0783e-01, PNorm = 53.9790, GNorm = 61.7302, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.530577
Epoch 255
Validation binary_cross_entropy = 0.526395
Epoch 256
Validation binary_cross_entropy = 0.524626
Epoch 257
Validation binary_cross_entropy = 0.518008
Epoch 258
Validation binary_cross_entropy = 0.506232
Epoch 259
Loss = 7.2296e-01, PNorm = 53.9846, GNorm = 58.8346, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.501972
Epoch 260
Validation binary_cross_entropy = 0.497709
Epoch 261
Validation binary_cross_entropy = 0.493053
Epoch 262
Validation binary_cross_entropy = 0.482440
Epoch 263
Validation binary_cross_entropy = 0.470818
Epoch 264
Loss = 5.9454e-01, PNorm = 53.9894, GNorm = 32.1599, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.467462
Epoch 265
Validation binary_cross_entropy = 0.452419
Epoch 266
Validation binary_cross_entropy = 0.442649
Epoch 267
Validation binary_cross_entropy = 0.470800
Epoch 268
Validation binary_cross_entropy = 0.498514
Epoch 269
Loss = 6.8541e-01, PNorm = 53.9939, GNorm = 41.3801, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.499763
Epoch 270
Validation binary_cross_entropy = 0.493288
Epoch 271
Validation binary_cross_entropy = 0.482567
Epoch 272
Validation binary_cross_entropy = 0.476737
Epoch 273
Validation binary_cross_entropy = 0.475351
Epoch 274
Loss = 5.3353e-01, PNorm = 53.9984, GNorm = 52.5071, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.469442
Epoch 275
Validation binary_cross_entropy = 0.475968
Epoch 276
Validation binary_cross_entropy = 0.482254
Epoch 277
Validation binary_cross_entropy = 0.482375
Epoch 278
Validation binary_cross_entropy = 0.481524
Epoch 279
Loss = 5.5373e-01, PNorm = 54.0034, GNorm = 29.4202, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.486015
Epoch 280
Validation binary_cross_entropy = 0.481868
Epoch 281
Validation binary_cross_entropy = 0.496519
Epoch 282
Validation binary_cross_entropy = 0.521732
Epoch 283
Validation binary_cross_entropy = 0.533265
Epoch 284
Loss = 7.4140e-01, PNorm = 54.0078, GNorm = 59.3206, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.538521
Epoch 285
Validation binary_cross_entropy = 0.538948
Epoch 286
Validation binary_cross_entropy = 0.483560
Epoch 287
Validation binary_cross_entropy = 0.446947
Epoch 288
Validation binary_cross_entropy = 0.451341
Epoch 289
Loss = 6.0827e-01, PNorm = 54.0132, GNorm = 30.5363, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.457531
Epoch 290
Validation binary_cross_entropy = 0.478137
Epoch 291
Validation binary_cross_entropy = 0.506860
Epoch 292
Validation binary_cross_entropy = 0.512488
Epoch 293
Validation binary_cross_entropy = 0.494539
Epoch 294
Loss = 4.5732e-01, PNorm = 54.0181, GNorm = 35.0409, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.479978
Epoch 295
Validation binary_cross_entropy = 0.468447
Epoch 296
Validation binary_cross_entropy = 0.484111
Epoch 297
Validation binary_cross_entropy = 0.498502
Epoch 298
Validation binary_cross_entropy = 0.520127
Epoch 299
Loss = 6.0620e-01, PNorm = 54.0220, GNorm = 50.4548, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.522632
Model 0 best validation binary_cross_entropy = 0.331497 on epoch 117
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.392099
Ensemble test binary_cross_entropy = 0.392099
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=300, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.35000000000000003, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 5,506,601
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 24.859888
Epoch 1
Validation binary_cross_entropy = 18.914782
Epoch 2
Validation binary_cross_entropy = 17.446230
Epoch 3
Validation binary_cross_entropy = 8.865360
Epoch 4
Loss = 1.0431e+02, PNorm = 52.2827, GNorm = 2280.4259, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 62.410423
Epoch 5
Validation binary_cross_entropy = 6.404084
Epoch 6
Validation binary_cross_entropy = 74.747696
Epoch 7
Validation binary_cross_entropy = 7.231612
Epoch 8
Validation binary_cross_entropy = 61.127575
Epoch 9
Loss = 1.4338e+02, PNorm = 52.4968, GNorm = 1328.5880, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 29.773788
Epoch 10
Validation binary_cross_entropy = 8.474654
Epoch 11
Validation binary_cross_entropy = 79.693573
Epoch 12
Validation binary_cross_entropy = 10.088241
Epoch 13
Validation binary_cross_entropy = 12.318732
Epoch 14
Loss = 5.1445e+01, PNorm = 52.7520, GNorm = 764.9008, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 33.608913
Epoch 15
Validation binary_cross_entropy = 10.823800
Epoch 16
Validation binary_cross_entropy = 8.954927
Epoch 17
Validation binary_cross_entropy = 24.265453
Epoch 18
Validation binary_cross_entropy = 15.532523
Epoch 19
Loss = 4.8180e+01, PNorm = 52.9456, GNorm = 473.8879, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 7.139801
Epoch 20
Validation binary_cross_entropy = 7.297338
Epoch 21
Validation binary_cross_entropy = 26.666771
Epoch 22
Validation binary_cross_entropy = 13.111572
Epoch 23
Validation binary_cross_entropy = 6.247180
Epoch 24
Loss = 4.4113e+01, PNorm = 53.0827, GNorm = 1228.7640, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 6.466504
Epoch 25
Validation binary_cross_entropy = 37.739689
Epoch 26
Validation binary_cross_entropy = 15.863431
Epoch 27
Validation binary_cross_entropy = 6.109457
Epoch 28
Validation binary_cross_entropy = 6.681278
Epoch 29
Loss = 3.5700e+01, PNorm = 53.1896, GNorm = 913.8607, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 23.795135
Epoch 30
Validation binary_cross_entropy = 7.469909
Epoch 31
Validation binary_cross_entropy = 6.038883
Epoch 32
Validation binary_cross_entropy = 5.481349
Epoch 33
Validation binary_cross_entropy = 28.825108
Epoch 34
Loss = 4.4431e+01, PNorm = 53.2767, GNorm = 1661.1527, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 7.406307
Epoch 35
Validation binary_cross_entropy = 4.943899
Epoch 36
Validation binary_cross_entropy = 5.101423
Epoch 37
Validation binary_cross_entropy = 26.969242
Epoch 38
Validation binary_cross_entropy = 29.273466
Epoch 39
Loss = 2.3642e+01, PNorm = 53.3488, GNorm = 651.7350, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 7.018126
Epoch 40
Validation binary_cross_entropy = 5.337154
Epoch 41
Validation binary_cross_entropy = 5.304592
Epoch 42
Validation binary_cross_entropy = 14.484010
Epoch 43
Validation binary_cross_entropy = 16.169886
Epoch 44
Loss = 1.3147e+01, PNorm = 53.4069, GNorm = 516.3068, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 9.253247
Epoch 45
Validation binary_cross_entropy = 6.626994
Epoch 46
Validation binary_cross_entropy = 6.406500
Epoch 47
Validation binary_cross_entropy = 8.331089
Epoch 48
Validation binary_cross_entropy = 5.814280
Epoch 49
Loss = 2.2136e+01, PNorm = 53.4562, GNorm = 430.1541, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 7.663727
Epoch 50
Validation binary_cross_entropy = 10.815032
Epoch 51
Validation binary_cross_entropy = 13.941514
Epoch 52
Validation binary_cross_entropy = 7.132361
Epoch 53
Validation binary_cross_entropy = 6.733698
Epoch 54
Loss = 1.2524e+01, PNorm = 53.4991, GNorm = 315.8481, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 20.733223
Epoch 55
Validation binary_cross_entropy = 29.120247
Epoch 56
Validation binary_cross_entropy = 7.683145
Epoch 57
Validation binary_cross_entropy = 5.743536
Epoch 58
Validation binary_cross_entropy = 10.149343
Epoch 59
Loss = 1.2872e+01, PNorm = 53.5427, GNorm = 374.8407, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 28.666849
Epoch 60
Validation binary_cross_entropy = 16.153223
Epoch 61
Validation binary_cross_entropy = 5.577737
Epoch 62
Validation binary_cross_entropy = 5.989872
Epoch 63
Validation binary_cross_entropy = 9.057140
Epoch 64
Loss = 5.8347e+00, PNorm = 53.5857, GNorm = 306.6684, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 13.515939
Epoch 65
Validation binary_cross_entropy = 12.329396
Epoch 66
Validation binary_cross_entropy = 13.624800
Epoch 67
Validation binary_cross_entropy = 10.153930
Epoch 68
Validation binary_cross_entropy = 8.011189
Epoch 69
Loss = 8.4883e+00, PNorm = 53.6238, GNorm = 220.7265, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 4.803281
Epoch 70
Validation binary_cross_entropy = 5.433155
Epoch 71
Validation binary_cross_entropy = 11.929615
Epoch 72
Validation binary_cross_entropy = 23.037399
Epoch 73
Validation binary_cross_entropy = 12.981843
Epoch 74
Loss = 6.6092e+00, PNorm = 53.6576, GNorm = 252.2007, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 4.396577
Epoch 75
Validation binary_cross_entropy = 3.693799
Epoch 76
Validation binary_cross_entropy = 5.920975
Epoch 77
Validation binary_cross_entropy = 3.275139
Epoch 78
Validation binary_cross_entropy = 2.554428
Epoch 79
Loss = 1.0724e+01, PNorm = 53.6877, GNorm = 210.1712, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 2.034095
Epoch 80
Validation binary_cross_entropy = 9.812684
Epoch 81
Validation binary_cross_entropy = 6.228921
Epoch 82
Validation binary_cross_entropy = 3.459965
Epoch 83
Validation binary_cross_entropy = 5.817109
Epoch 84
Loss = 7.5470e+00, PNorm = 53.7170, GNorm = 248.8911, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 3.601533
Epoch 85
Validation binary_cross_entropy = 3.493377
Epoch 86
Validation binary_cross_entropy = 6.132893
Epoch 87
Validation binary_cross_entropy = 13.352379
Epoch 88
Validation binary_cross_entropy = 6.980868
Epoch 89
Loss = 3.5891e+00, PNorm = 53.7453, GNorm = 113.1783, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 3.850175
Epoch 90
Validation binary_cross_entropy = 4.140520
Epoch 91
Validation binary_cross_entropy = 6.439114
Epoch 92
Validation binary_cross_entropy = 7.973867
Epoch 93
Validation binary_cross_entropy = 6.228463
Epoch 94
Loss = 4.6106e+00, PNorm = 53.7702, GNorm = 214.8104, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 2.693380
Epoch 95
Validation binary_cross_entropy = 2.584130
Epoch 96
Validation binary_cross_entropy = 4.558208
Epoch 97
Validation binary_cross_entropy = 8.912696
Epoch 98
Validation binary_cross_entropy = 16.585526
Epoch 99
Loss = 7.5740e+00, PNorm = 53.7927, GNorm = 484.7529, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 4.261343
Epoch 100
Validation binary_cross_entropy = 1.406801
Epoch 101
Validation binary_cross_entropy = 1.484224
Epoch 102
Validation binary_cross_entropy = 1.497913
Epoch 103
Validation binary_cross_entropy = 5.515235
Epoch 104
Loss = 5.1076e+00, PNorm = 53.8169, GNorm = 254.0248, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 11.309655
Epoch 105
Validation binary_cross_entropy = 5.447124
Epoch 106
Validation binary_cross_entropy = 1.737543
Epoch 107
Validation binary_cross_entropy = 1.349924
Epoch 108
Validation binary_cross_entropy = 1.464856
Epoch 109
Loss = 2.8563e+00, PNorm = 53.8433, GNorm = 134.3585, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 3.370164
Epoch 110
Validation binary_cross_entropy = 3.797599
Epoch 111
Validation binary_cross_entropy = 1.693756
Epoch 112
Validation binary_cross_entropy = 1.293514
Epoch 113
Validation binary_cross_entropy = 1.220839
Epoch 114
Loss = 4.1532e+00, PNorm = 53.8683, GNorm = 131.0973, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 1.814878
Epoch 115
Validation binary_cross_entropy = 2.820646
Epoch 116
Validation binary_cross_entropy = 1.992201
Epoch 117
Validation binary_cross_entropy = 1.516917
Epoch 118
Validation binary_cross_entropy = 1.009643
Epoch 119
Loss = 1.8614e+00, PNorm = 53.8926, GNorm = 126.3169, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 2.056792
Epoch 120
Validation binary_cross_entropy = 4.051142
Epoch 121
Validation binary_cross_entropy = 3.120163
Epoch 122
Validation binary_cross_entropy = 0.879672
Epoch 123
Validation binary_cross_entropy = 0.814329
Epoch 124
Loss = 3.3606e+00, PNorm = 53.9142, GNorm = 260.7290, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.775379
Epoch 125
Validation binary_cross_entropy = 0.748777
Epoch 126
Validation binary_cross_entropy = 1.023987
Epoch 127
Validation binary_cross_entropy = 1.628263
Epoch 128
Validation binary_cross_entropy = 1.956604
Epoch 129
Loss = 2.5129e+00, PNorm = 53.9352, GNorm = 87.1697, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 1.469919
Epoch 130
Validation binary_cross_entropy = 1.071860
Epoch 131
Validation binary_cross_entropy = 0.890386
Epoch 132
Validation binary_cross_entropy = 0.902335
Epoch 133
Validation binary_cross_entropy = 0.922168
Epoch 134
Loss = 9.8793e-01, PNorm = 53.9537, GNorm = 73.3706, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 1.385422
Epoch 135
Validation binary_cross_entropy = 1.705504
Epoch 136
Validation binary_cross_entropy = 1.623240
Epoch 137
Validation binary_cross_entropy = 1.432956
Epoch 138
Validation binary_cross_entropy = 1.275490
Epoch 139
Loss = 2.6643e+00, PNorm = 53.9689, GNorm = 149.9362, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 1.262088
Epoch 140
Validation binary_cross_entropy = 1.413416
Epoch 141
Validation binary_cross_entropy = 1.883512
Epoch 142
Validation binary_cross_entropy = 2.130470
Epoch 143
Validation binary_cross_entropy = 2.114132
Epoch 144
Loss = 2.3630e+00, PNorm = 53.9833, GNorm = 124.4332, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 1.226630
Epoch 145
Validation binary_cross_entropy = 0.767760
Epoch 146
Validation binary_cross_entropy = 0.727667
Epoch 147
Validation binary_cross_entropy = 1.016215
Epoch 148
Validation binary_cross_entropy = 1.550247
Epoch 149
Loss = 2.0729e+00, PNorm = 53.9976, GNorm = 124.5892, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 1.749849
Epoch 150
Validation binary_cross_entropy = 1.711003
Epoch 151
Validation binary_cross_entropy = 1.287400
Epoch 152
Validation binary_cross_entropy = 1.003247
Epoch 153
Validation binary_cross_entropy = 0.809593
Epoch 154
Loss = 2.6236e+00, PNorm = 54.0121, GNorm = 127.8645, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.807795
Epoch 155
Validation binary_cross_entropy = 0.958248
Epoch 156
Validation binary_cross_entropy = 1.112982
Epoch 157
Validation binary_cross_entropy = 1.327810
Epoch 158
Validation binary_cross_entropy = 0.993264
Epoch 159
Loss = 1.7027e+00, PNorm = 54.0244, GNorm = 85.0430, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.796269
Epoch 160
Validation binary_cross_entropy = 0.610020
Epoch 161
Validation binary_cross_entropy = 0.580350
Epoch 162
Validation binary_cross_entropy = 0.664804
Epoch 163
Validation binary_cross_entropy = 0.802548
Epoch 164
Loss = 1.2695e+00, PNorm = 54.0345, GNorm = 103.6997, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.881843
Epoch 165
Validation binary_cross_entropy = 0.951686
Epoch 166
Validation binary_cross_entropy = 0.924424
Epoch 167
Validation binary_cross_entropy = 0.659266
Epoch 168
Validation binary_cross_entropy = 0.646495
Epoch 169
Loss = 1.4392e+00, PNorm = 54.0466, GNorm = 81.9588, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.797320
Epoch 170
Validation binary_cross_entropy = 1.185233
Epoch 171
Validation binary_cross_entropy = 1.573597
Epoch 172
Validation binary_cross_entropy = 1.747539
Epoch 173
Validation binary_cross_entropy = 1.181988
Epoch 174
Loss = 1.4510e+00, PNorm = 54.0580, GNorm = 68.4850, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 1.020984
Epoch 175
Validation binary_cross_entropy = 0.892006
Epoch 176
Validation binary_cross_entropy = 0.832407
Epoch 177
Validation binary_cross_entropy = 0.839925
Epoch 178
Validation binary_cross_entropy = 0.913767
Epoch 179
Loss = 8.9936e-01, PNorm = 54.0689, GNorm = 83.9456, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 1.049039
Epoch 180
Validation binary_cross_entropy = 1.122457
Epoch 181
Validation binary_cross_entropy = 1.157838
Epoch 182
Validation binary_cross_entropy = 1.014225
Epoch 183
Validation binary_cross_entropy = 0.900006
Epoch 184
Loss = 1.8511e+00, PNorm = 54.0784, GNorm = 105.1090, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.769043
Epoch 185
Validation binary_cross_entropy = 0.695798
Epoch 186
Validation binary_cross_entropy = 0.683022
Epoch 187
Validation binary_cross_entropy = 0.695888
Epoch 188
Validation binary_cross_entropy = 0.742839
Epoch 189
Loss = 1.2637e+00, PNorm = 54.0874, GNorm = 64.8850, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.857713
Epoch 190
Validation binary_cross_entropy = 0.947802
Epoch 191
Validation binary_cross_entropy = 0.933719
Epoch 192
Validation binary_cross_entropy = 0.851105
Epoch 193
Validation binary_cross_entropy = 0.660339
Epoch 194
Loss = 8.8102e-01, PNorm = 54.0958, GNorm = 103.1248, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.558794
Epoch 195
Validation binary_cross_entropy = 0.528747
Epoch 196
Validation binary_cross_entropy = 0.536848
Epoch 197
Validation binary_cross_entropy = 0.629094
Epoch 198
Validation binary_cross_entropy = 0.791155
Epoch 199
Loss = 1.2506e+00, PNorm = 54.1048, GNorm = 107.0981, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.868540
Epoch 200
Validation binary_cross_entropy = 0.738561
Epoch 201
Validation binary_cross_entropy = 0.638178
Epoch 202
Validation binary_cross_entropy = 0.591757
Epoch 203
Validation binary_cross_entropy = 0.538130
Epoch 204
Loss = 6.4431e-01, PNorm = 54.1142, GNorm = 88.3809, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.500386
Epoch 205
Validation binary_cross_entropy = 0.480576
Epoch 206
Validation binary_cross_entropy = 0.485887
Epoch 207
Validation binary_cross_entropy = 0.490971
Epoch 208
Validation binary_cross_entropy = 0.519444
Epoch 209
Loss = 7.2466e-01, PNorm = 54.1225, GNorm = 49.1668, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.572226
Epoch 210
Validation binary_cross_entropy = 0.639066
Epoch 211
Validation binary_cross_entropy = 0.708211
Epoch 212
Validation binary_cross_entropy = 0.652720
Epoch 213
Validation binary_cross_entropy = 0.566284
Epoch 214
Loss = 9.5463e-01, PNorm = 54.1294, GNorm = 55.3829, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.527162
Epoch 215
Validation binary_cross_entropy = 0.533683
Epoch 216
Validation binary_cross_entropy = 0.539884
Epoch 217
Validation binary_cross_entropy = 0.570701
Epoch 218
Validation binary_cross_entropy = 0.607104
Epoch 219
Loss = 5.6794e-01, PNorm = 54.1351, GNorm = 38.9845, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.644345
Epoch 220
Validation binary_cross_entropy = 0.612829
Epoch 221
Validation binary_cross_entropy = 0.633098
Epoch 222
Validation binary_cross_entropy = 0.620437
Epoch 223
Validation binary_cross_entropy = 0.571608
Epoch 224
Loss = 7.0601e-01, PNorm = 54.1406, GNorm = 61.8622, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.550019
Epoch 225
Validation binary_cross_entropy = 0.557565
Epoch 226
Validation binary_cross_entropy = 0.514754
Epoch 227
Validation binary_cross_entropy = 0.528096
Epoch 228
Validation binary_cross_entropy = 0.576037
Epoch 229
Loss = 6.8471e-01, PNorm = 54.1464, GNorm = 126.1776, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.635958
Epoch 230
Validation binary_cross_entropy = 0.561174
Epoch 231
Validation binary_cross_entropy = 0.515394
Epoch 232
Validation binary_cross_entropy = 0.516652
Epoch 233
Validation binary_cross_entropy = 0.582087
Epoch 234
Loss = 6.5025e-01, PNorm = 54.1532, GNorm = 73.5937, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.612386
Epoch 235
Validation binary_cross_entropy = 0.592580
Epoch 236
Validation binary_cross_entropy = 0.572423
Epoch 237
Validation binary_cross_entropy = 0.547731
Epoch 238
Validation binary_cross_entropy = 0.530290
Epoch 239
Loss = 1.0275e+00, PNorm = 54.1600, GNorm = 68.9384, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.496510
Epoch 240
Validation binary_cross_entropy = 0.478121
Epoch 241
Validation binary_cross_entropy = 0.484046
Epoch 242
Validation binary_cross_entropy = 0.515713
Epoch 243
Validation binary_cross_entropy = 0.526534
Epoch 244
Loss = 7.0743e-01, PNorm = 54.1663, GNorm = 55.0056, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.534165
Epoch 245
Validation binary_cross_entropy = 0.543104
Epoch 246
Validation binary_cross_entropy = 0.556664
Epoch 247
Validation binary_cross_entropy = 0.550498
Epoch 248
Validation binary_cross_entropy = 0.534833
Epoch 249
Loss = 7.8610e-01, PNorm = 54.1722, GNorm = 75.0031, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.523757
Epoch 250
Validation binary_cross_entropy = 0.498287
Epoch 251
Validation binary_cross_entropy = 0.468154
Epoch 252
Validation binary_cross_entropy = 0.459002
Epoch 253
Validation binary_cross_entropy = 0.459709
Epoch 254
Loss = 9.4145e-01, PNorm = 54.1778, GNorm = 78.1701, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.483581
Epoch 255
Validation binary_cross_entropy = 0.526051
Epoch 256
Validation binary_cross_entropy = 0.553210
Epoch 257
Validation binary_cross_entropy = 0.587133
Epoch 258
Validation binary_cross_entropy = 0.585375
Epoch 259
Loss = 5.3965e-01, PNorm = 54.1835, GNorm = 38.5489, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.569872
Epoch 260
Validation binary_cross_entropy = 0.524321
Epoch 261
Validation binary_cross_entropy = 0.436719
Epoch 262
Validation binary_cross_entropy = 0.402831
Epoch 263
Validation binary_cross_entropy = 0.410245
Epoch 264
Loss = 7.5744e-01, PNorm = 54.1892, GNorm = 62.4243, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.444403
Epoch 265
Validation binary_cross_entropy = 0.500508
Epoch 266
Validation binary_cross_entropy = 0.568306
Epoch 267
Validation binary_cross_entropy = 0.582119
Epoch 268
Validation binary_cross_entropy = 0.563617
Epoch 269
Loss = 7.3416e-01, PNorm = 54.1948, GNorm = 89.4266, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.495442
Epoch 270
Validation binary_cross_entropy = 0.439629
Epoch 271
Validation binary_cross_entropy = 0.428347
Epoch 272
Validation binary_cross_entropy = 0.440874
Epoch 273
Validation binary_cross_entropy = 0.455542
Epoch 274
Loss = 3.5670e-01, PNorm = 54.2007, GNorm = 35.5526, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.475307
Epoch 275
Validation binary_cross_entropy = 0.517379
Epoch 276
Validation binary_cross_entropy = 0.560053
Epoch 277
Validation binary_cross_entropy = 0.580391
Epoch 278
Validation binary_cross_entropy = 0.577492
Epoch 279
Loss = 6.2964e-01, PNorm = 54.2060, GNorm = 82.5028, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.568422
Epoch 280
Validation binary_cross_entropy = 0.557471
Epoch 281
Validation binary_cross_entropy = 0.545432
Epoch 282
Validation binary_cross_entropy = 0.516924
Epoch 283
Validation binary_cross_entropy = 0.487500
Epoch 284
Loss = 8.9138e-01, PNorm = 54.2110, GNorm = 68.0477, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.467752
Epoch 285
Validation binary_cross_entropy = 0.458081
Epoch 286
Validation binary_cross_entropy = 0.473572
Epoch 287
Validation binary_cross_entropy = 0.500894
Epoch 288
Validation binary_cross_entropy = 0.516422
Epoch 289
Loss = 4.9453e-01, PNorm = 54.2164, GNorm = 41.4405, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.533451
Epoch 290
Validation binary_cross_entropy = 0.532683
Epoch 291
Validation binary_cross_entropy = 0.530982
Epoch 292
Validation binary_cross_entropy = 0.534686
Epoch 293
Validation binary_cross_entropy = 0.545562
Epoch 294
Loss = 8.6309e-01, PNorm = 54.2217, GNorm = 28.5218, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.556855
Epoch 295
Validation binary_cross_entropy = 0.546991
Epoch 296
Validation binary_cross_entropy = 0.542116
Epoch 297
Validation binary_cross_entropy = 0.545092
Epoch 298
Validation binary_cross_entropy = 0.548751
Epoch 299
Loss = 7.6810e-01, PNorm = 54.2266, GNorm = 57.6126, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.560436
Model 0 best validation binary_cross_entropy = 0.402831 on epoch 262
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.428621
Ensemble test binary_cross_entropy = 0.428621
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=300, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.35000000000000003, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 5,506,601
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 81.557243
Epoch 1
Validation binary_cross_entropy = 7.034471
Epoch 2
Validation binary_cross_entropy = 86.731430
Epoch 3
Validation binary_cross_entropy = 14.474064
Epoch 4
Loss = 1.3485e+02, PNorm = 52.2788, GNorm = 2484.2335, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 8.997748
Epoch 5
Validation binary_cross_entropy = 27.935839
Epoch 6
Validation binary_cross_entropy = 54.641949
Epoch 7
Validation binary_cross_entropy = 9.883725
Epoch 8
Validation binary_cross_entropy = 7.738192
Epoch 9
Loss = 1.0376e+02, PNorm = 52.4743, GNorm = 2031.7190, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 68.531525
Epoch 10
Validation binary_cross_entropy = 39.516026
Epoch 11
Validation binary_cross_entropy = 7.399543
Epoch 12
Validation binary_cross_entropy = 66.928864
Epoch 13
Validation binary_cross_entropy = 47.574677
Epoch 14
Loss = 6.9356e+01, PNorm = 52.7061, GNorm = 1729.3014, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 7.220512
Epoch 15
Validation binary_cross_entropy = 63.136173
Epoch 16
Validation binary_cross_entropy = 19.794876
Epoch 17
Validation binary_cross_entropy = 6.371583
Epoch 18
Validation binary_cross_entropy = 14.379211
Epoch 19
Loss = 4.8672e+01, PNorm = 52.8864, GNorm = 1043.7052, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 26.748283
Epoch 20
Validation binary_cross_entropy = 7.556273
Epoch 21
Validation binary_cross_entropy = 9.135886
Epoch 22
Validation binary_cross_entropy = 18.305515
Epoch 23
Validation binary_cross_entropy = 7.373621
Epoch 24
Loss = 3.6963e+01, PNorm = 53.0176, GNorm = 1183.3344, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 8.233811
Epoch 25
Validation binary_cross_entropy = 44.368374
Epoch 26
Validation binary_cross_entropy = 26.291534
Epoch 27
Validation binary_cross_entropy = 8.006796
Epoch 28
Validation binary_cross_entropy = 7.216892
Epoch 29
Loss = 2.4569e+01, PNorm = 53.1139, GNorm = 652.1892, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 18.482231
Epoch 30
Validation binary_cross_entropy = 25.013100
Epoch 31
Validation binary_cross_entropy = 12.439795
Epoch 32
Validation binary_cross_entropy = 7.285511
Epoch 33
Validation binary_cross_entropy = 5.949424
Epoch 34
Loss = 7.7867e+00, PNorm = 53.1862, GNorm = 297.8949, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 16.907104
Epoch 35
Validation binary_cross_entropy = 17.794895
Epoch 36
Validation binary_cross_entropy = 6.260186
Epoch 37
Validation binary_cross_entropy = 4.896551
Epoch 38
Validation binary_cross_entropy = 5.900694
Epoch 39
Loss = 1.3137e+01, PNorm = 53.2439, GNorm = 819.1746, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 8.164300
Epoch 40
Validation binary_cross_entropy = 15.131311
Epoch 41
Validation binary_cross_entropy = 6.182682
Epoch 42
Validation binary_cross_entropy = 6.906587
Epoch 43
Validation binary_cross_entropy = 5.800202
Epoch 44
Loss = 2.1342e+01, PNorm = 53.2954, GNorm = 807.9530, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 5.834361
Epoch 45
Validation binary_cross_entropy = 8.879105
Epoch 46
Validation binary_cross_entropy = 29.142893
Epoch 47
Validation binary_cross_entropy = 28.843271
Epoch 48
Validation binary_cross_entropy = 4.548599
Epoch 49
Loss = 2.0812e+01, PNorm = 53.3444, GNorm = 432.9100, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 7.562689
Epoch 50
Validation binary_cross_entropy = 25.981874
Epoch 51
Validation binary_cross_entropy = 17.724512
Epoch 52
Validation binary_cross_entropy = 5.936898
Epoch 53
Validation binary_cross_entropy = 5.600357
Epoch 54
Loss = 1.2308e+01, PNorm = 53.3924, GNorm = 348.1620, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 19.449825
Epoch 55
Validation binary_cross_entropy = 27.957146
Epoch 56
Validation binary_cross_entropy = 9.647442
Epoch 57
Validation binary_cross_entropy = 3.650902
Epoch 58
Validation binary_cross_entropy = 3.510438
Epoch 59
Loss = 9.1384e+00, PNorm = 53.4352, GNorm = 228.5429, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 15.516525
Epoch 60
Validation binary_cross_entropy = 43.478474
Epoch 61
Validation binary_cross_entropy = 23.765066
Epoch 62
Validation binary_cross_entropy = 3.142723
Epoch 63
Validation binary_cross_entropy = 2.434478
Epoch 64
Loss = 1.0649e+01, PNorm = 53.4745, GNorm = 257.8490, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 4.225229
Epoch 65
Validation binary_cross_entropy = 4.288207
Epoch 66
Validation binary_cross_entropy = 6.120552
Epoch 67
Validation binary_cross_entropy = 4.124246
Epoch 68
Validation binary_cross_entropy = 3.578854
Epoch 69
Loss = 4.4666e+00, PNorm = 53.5090, GNorm = 112.9575, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 2.373564
Epoch 70
Validation binary_cross_entropy = 3.920187
Epoch 71
Validation binary_cross_entropy = 4.527263
Epoch 72
Validation binary_cross_entropy = 1.582765
Epoch 73
Validation binary_cross_entropy = 1.516694
Epoch 74
Loss = 4.7352e+00, PNorm = 53.5386, GNorm = 217.3596, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 2.636980
Epoch 75
Validation binary_cross_entropy = 2.917152
Epoch 76
Validation binary_cross_entropy = 2.009979
Epoch 77
Validation binary_cross_entropy = 2.122139
Epoch 78
Validation binary_cross_entropy = 6.205204
Epoch 79
Loss = 3.1355e+00, PNorm = 53.5680, GNorm = 276.3781, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 12.362061
Epoch 80
Validation binary_cross_entropy = 3.631478
Epoch 81
Validation binary_cross_entropy = 1.364373
Epoch 82
Validation binary_cross_entropy = 1.361525
Epoch 83
Validation binary_cross_entropy = 1.240796
Epoch 84
Loss = 4.3801e+00, PNorm = 53.5962, GNorm = 212.5326, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 1.905283
Epoch 85
Validation binary_cross_entropy = 2.323967
Epoch 86
Validation binary_cross_entropy = 1.981194
Epoch 87
Validation binary_cross_entropy = 1.862757
Epoch 88
Validation binary_cross_entropy = 1.791387
Epoch 89
Loss = 8.1241e+00, PNorm = 53.6208, GNorm = 202.2975, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 1.916802
Epoch 90
Validation binary_cross_entropy = 2.411136
Epoch 91
Validation binary_cross_entropy = 2.834367
Epoch 92
Validation binary_cross_entropy = 0.908733
Epoch 93
Validation binary_cross_entropy = 1.074625
Epoch 94
Loss = 3.0175e+00, PNorm = 53.6418, GNorm = 216.8893, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 1.034239
Epoch 95
Validation binary_cross_entropy = 1.293092
Epoch 96
Validation binary_cross_entropy = 3.470495
Epoch 97
Validation binary_cross_entropy = 3.656750
Epoch 98
Validation binary_cross_entropy = 1.542149
Epoch 99
Loss = 2.0078e+00, PNorm = 53.6620, GNorm = 71.4107, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.711815
Epoch 100
Validation binary_cross_entropy = 0.563132
Epoch 101
Validation binary_cross_entropy = 0.569171
Epoch 102
Validation binary_cross_entropy = 0.604944
Epoch 103
Validation binary_cross_entropy = 0.616525
Epoch 104
Loss = 2.4266e+00, PNorm = 53.6805, GNorm = 116.3971, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.588642
Epoch 105
Validation binary_cross_entropy = 0.620753
Epoch 106
Validation binary_cross_entropy = 0.591343
Epoch 107
Validation binary_cross_entropy = 0.598385
Epoch 108
Validation binary_cross_entropy = 0.587360
Epoch 109
Loss = 1.5250e+00, PNorm = 53.6996, GNorm = 68.9909, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.585362
Epoch 110
Validation binary_cross_entropy = 0.597122
Epoch 111
Validation binary_cross_entropy = 0.610017
Epoch 112
Validation binary_cross_entropy = 0.618280
Epoch 113
Validation binary_cross_entropy = 0.634644
Epoch 114
Loss = 1.7351e+00, PNorm = 53.7158, GNorm = 84.9077, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.672462
Epoch 115
Validation binary_cross_entropy = 0.674419
Epoch 116
Validation binary_cross_entropy = 0.766456
Epoch 117
Validation binary_cross_entropy = 0.852469
Epoch 118
Validation binary_cross_entropy = 0.774645
Epoch 119
Loss = 1.5955e+00, PNorm = 53.7309, GNorm = 64.6536, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.721168
Epoch 120
Validation binary_cross_entropy = 0.706896
Epoch 121
Validation binary_cross_entropy = 0.720177
Epoch 122
Validation binary_cross_entropy = 0.884747
Epoch 123
Validation binary_cross_entropy = 1.190724
Epoch 124
Loss = 2.7734e+00, PNorm = 53.7447, GNorm = 247.0415, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.946075
Epoch 125
Validation binary_cross_entropy = 0.625548
Epoch 126
Validation binary_cross_entropy = 0.572535
Epoch 127
Validation binary_cross_entropy = 0.576693
Epoch 128
Validation binary_cross_entropy = 0.553261
Epoch 129
Loss = 1.1474e+00, PNorm = 53.7574, GNorm = 81.5399, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.528877
Epoch 130
Validation binary_cross_entropy = 0.606676
Epoch 131
Validation binary_cross_entropy = 0.652124
Epoch 132
Validation binary_cross_entropy = 0.480665
Epoch 133
Validation binary_cross_entropy = 0.418385
Epoch 134
Loss = 1.0902e+00, PNorm = 53.7702, GNorm = 98.8601, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.397199
Epoch 135
Validation binary_cross_entropy = 0.399699
Epoch 136
Validation binary_cross_entropy = 0.423675
Epoch 137
Validation binary_cross_entropy = 0.447809
Epoch 138
Validation binary_cross_entropy = 0.463242
Epoch 139
Loss = 1.1129e+00, PNorm = 53.7825, GNorm = 66.7677, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.458423
Epoch 140
Validation binary_cross_entropy = 0.488941
Epoch 141
Validation binary_cross_entropy = 0.559238
Epoch 142
Validation binary_cross_entropy = 0.607388
Epoch 143
Validation binary_cross_entropy = 0.648750
Epoch 144
Loss = 7.8547e-01, PNorm = 53.7932, GNorm = 112.2611, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.630911
Epoch 145
Validation binary_cross_entropy = 0.564962
Epoch 146
Validation binary_cross_entropy = 0.568732
Epoch 147
Validation binary_cross_entropy = 0.541389
Epoch 148
Validation binary_cross_entropy = 0.522484
Epoch 149
Loss = 6.8249e-01, PNorm = 53.8027, GNorm = 82.3239, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.521887
Epoch 150
Validation binary_cross_entropy = 0.543425
Epoch 151
Validation binary_cross_entropy = 0.572815
Epoch 152
Validation binary_cross_entropy = 0.563627
Epoch 153
Validation binary_cross_entropy = 0.550422
Epoch 154
Loss = 7.7779e-01, PNorm = 53.8113, GNorm = 50.4875, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.538253
Epoch 155
Validation binary_cross_entropy = 0.587958
Epoch 156
Validation binary_cross_entropy = 0.594699
Epoch 157
Validation binary_cross_entropy = 0.637822
Epoch 158
Validation binary_cross_entropy = 0.666088
Epoch 159
Loss = 1.7291e+00, PNorm = 53.8187, GNorm = 229.3747, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.537375
Epoch 160
Validation binary_cross_entropy = 0.446296
Epoch 161
Validation binary_cross_entropy = 0.396979
Epoch 162
Validation binary_cross_entropy = 0.393512
Epoch 163
Validation binary_cross_entropy = 0.433529
Epoch 164
Loss = 1.1512e+00, PNorm = 53.8272, GNorm = 145.7384, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.535881
Epoch 165
Validation binary_cross_entropy = 0.763247
Epoch 166
Validation binary_cross_entropy = 0.837063
Epoch 167
Validation binary_cross_entropy = 0.610516
Epoch 168
Validation binary_cross_entropy = 0.497284
Epoch 169
Loss = 5.9186e-01, PNorm = 53.8361, GNorm = 44.2379, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.443459
Epoch 170
Validation binary_cross_entropy = 0.424009
Epoch 171
Validation binary_cross_entropy = 0.417666
Epoch 172
Validation binary_cross_entropy = 0.429981
Epoch 173
Validation binary_cross_entropy = 0.472716
Epoch 174
Loss = 8.3342e-01, PNorm = 53.8451, GNorm = 79.9916, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.513252
Epoch 175
Validation binary_cross_entropy = 0.510998
Epoch 176
Validation binary_cross_entropy = 0.509788
Epoch 177
Validation binary_cross_entropy = 0.497509
Epoch 178
Validation binary_cross_entropy = 0.491761
Epoch 179
Loss = 9.0853e-01, PNorm = 53.8535, GNorm = 63.8208, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.513494
Epoch 180
Validation binary_cross_entropy = 0.451372
Epoch 181
Validation binary_cross_entropy = 0.450021
Epoch 182
Validation binary_cross_entropy = 0.488644
Epoch 183
Validation binary_cross_entropy = 0.545915
Epoch 184
Loss = 8.2454e-01, PNorm = 53.8614, GNorm = 55.8901, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.628472
Epoch 185
Validation binary_cross_entropy = 0.768424
Epoch 186
Validation binary_cross_entropy = 0.859801
Epoch 187
Validation binary_cross_entropy = 0.818504
Epoch 188
Validation binary_cross_entropy = 0.566706
Epoch 189
Loss = 9.2912e-01, PNorm = 53.8686, GNorm = 43.9648, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.375790
Epoch 190
Validation binary_cross_entropy = 0.346104
Epoch 191
Validation binary_cross_entropy = 0.390804
Epoch 192
Validation binary_cross_entropy = 0.612793
Epoch 193
Validation binary_cross_entropy = 0.746524
Epoch 194
Loss = 1.2538e+00, PNorm = 53.8759, GNorm = 70.6751, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.602945
Epoch 195
Validation binary_cross_entropy = 0.527523
Epoch 196
Validation binary_cross_entropy = 0.465745
Epoch 197
Validation binary_cross_entropy = 0.409765
Epoch 198
Validation binary_cross_entropy = 0.392479
Epoch 199
Loss = 6.8680e-01, PNorm = 53.8829, GNorm = 57.8067, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.384827
Epoch 200
Validation binary_cross_entropy = 0.388402
Epoch 201
Validation binary_cross_entropy = 0.415939
Epoch 202
Validation binary_cross_entropy = 0.435067
Epoch 203
Validation binary_cross_entropy = 0.473878
Epoch 204
Loss = 9.3045e-01, PNorm = 53.8894, GNorm = 71.3625, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.506293
Epoch 205
Validation binary_cross_entropy = 0.477608
Epoch 206
Validation binary_cross_entropy = 0.438639
Epoch 207
Validation binary_cross_entropy = 0.411399
Epoch 208
Validation binary_cross_entropy = 0.400559
Epoch 209
Loss = 9.5044e-01, PNorm = 53.8967, GNorm = 58.0787, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.402283
Epoch 210
Validation binary_cross_entropy = 0.420509
Epoch 211
Validation binary_cross_entropy = 0.440331
Epoch 212
Validation binary_cross_entropy = 0.451360
Epoch 213
Validation binary_cross_entropy = 0.425677
Epoch 214
Loss = 1.1002e+00, PNorm = 53.9036, GNorm = 68.9169, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.439662
Epoch 215
Validation binary_cross_entropy = 0.454141
Epoch 216
Validation binary_cross_entropy = 0.447347
Epoch 217
Validation binary_cross_entropy = 0.422309
Epoch 218
Validation binary_cross_entropy = 0.401121
Epoch 219
Loss = 6.5710e-01, PNorm = 53.9095, GNorm = 49.4433, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.398453
Epoch 220
Validation binary_cross_entropy = 0.400726
Epoch 221
Validation binary_cross_entropy = 0.438059
Epoch 222
Validation binary_cross_entropy = 0.502047
Epoch 223
Validation binary_cross_entropy = 0.533600
Epoch 224
Loss = 6.6805e-01, PNorm = 53.9147, GNorm = 42.6234, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.509932
Epoch 225
Validation binary_cross_entropy = 0.449555
Epoch 226
Validation binary_cross_entropy = 0.380053
Epoch 227
Validation binary_cross_entropy = 0.338327
Epoch 228
Validation binary_cross_entropy = 0.343423
Epoch 229
Loss = 6.0148e-01, PNorm = 53.9200, GNorm = 54.4823, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.372514
Epoch 230
Validation binary_cross_entropy = 0.428752
Epoch 231
Validation binary_cross_entropy = 0.460134
Epoch 232
Validation binary_cross_entropy = 0.491012
Epoch 233
Validation binary_cross_entropy = 0.527465
Epoch 234
Loss = 5.7144e-01, PNorm = 53.9255, GNorm = 40.6600, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.559495
Epoch 235
Validation binary_cross_entropy = 0.583331
Epoch 236
Validation binary_cross_entropy = 0.650370
Epoch 237
Validation binary_cross_entropy = 0.559358
Epoch 238
Validation binary_cross_entropy = 0.481940
Epoch 239
Loss = 5.1796e-01, PNorm = 53.9305, GNorm = 39.4429, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.458817
Epoch 240
Validation binary_cross_entropy = 0.456973
Epoch 241
Validation binary_cross_entropy = 0.478596
Epoch 242
Validation binary_cross_entropy = 0.504200
Epoch 243
Validation binary_cross_entropy = 0.523725
Epoch 244
Loss = 8.4662e-01, PNorm = 53.9357, GNorm = 79.9032, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.563160
Epoch 245
Validation binary_cross_entropy = 0.571944
Epoch 246
Validation binary_cross_entropy = 0.567377
Epoch 247
Validation binary_cross_entropy = 0.559411
Epoch 248
Validation binary_cross_entropy = 0.576797
Epoch 249
Loss = 5.1142e-01, PNorm = 53.9402, GNorm = 40.8586, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.585687
Epoch 250
Validation binary_cross_entropy = 0.578113
Epoch 251
Validation binary_cross_entropy = 0.537661
Epoch 252
Validation binary_cross_entropy = 0.510413
Epoch 253
Validation binary_cross_entropy = 0.492350
Epoch 254
Loss = 4.8558e-01, PNorm = 53.9440, GNorm = 36.0377, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.483981
Epoch 255
Validation binary_cross_entropy = 0.478355
Epoch 256
Validation binary_cross_entropy = 0.471108
Epoch 257
Validation binary_cross_entropy = 0.467683
Epoch 258
Validation binary_cross_entropy = 0.468299
Epoch 259
Loss = 8.6378e-01, PNorm = 53.9476, GNorm = 66.1958, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.479020
Epoch 260
Validation binary_cross_entropy = 0.473706
Epoch 261
Validation binary_cross_entropy = 0.480093
Epoch 262
Validation binary_cross_entropy = 0.474728
Epoch 263
Validation binary_cross_entropy = 0.475327
Epoch 264
Loss = 4.7642e-01, PNorm = 53.9512, GNorm = 49.0981, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.482323
Epoch 265
Validation binary_cross_entropy = 0.487818
Epoch 266
Validation binary_cross_entropy = 0.498442
Epoch 267
Validation binary_cross_entropy = 0.497261
Epoch 268
Validation binary_cross_entropy = 0.488419
Epoch 269
Loss = 4.4970e-01, PNorm = 53.9549, GNorm = 24.3031, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.480060
Epoch 270
Validation binary_cross_entropy = 0.470623
Epoch 271
Validation binary_cross_entropy = 0.448935
Epoch 272
Validation binary_cross_entropy = 0.427133
Epoch 273
Validation binary_cross_entropy = 0.412936
Epoch 274
Loss = 4.5718e-01, PNorm = 53.9584, GNorm = 36.8899, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.446878
Epoch 275
Validation binary_cross_entropy = 0.482883
Epoch 276
Validation binary_cross_entropy = 0.538388
Epoch 277
Validation binary_cross_entropy = 0.596499
Epoch 278
Validation binary_cross_entropy = 0.612758
Epoch 279
Loss = 6.9975e-01, PNorm = 53.9623, GNorm = 57.5662, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.630938
Epoch 280
Validation binary_cross_entropy = 0.534798
Epoch 281
Validation binary_cross_entropy = 0.468926
Epoch 282
Validation binary_cross_entropy = 0.442001
Epoch 283
Validation binary_cross_entropy = 0.432688
Epoch 284
Loss = 6.3640e-01, PNorm = 53.9659, GNorm = 60.2790, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.444232
Epoch 285
Validation binary_cross_entropy = 0.470490
Epoch 286
Validation binary_cross_entropy = 0.492071
Epoch 287
Validation binary_cross_entropy = 0.509820
Epoch 288
Validation binary_cross_entropy = 0.520804
Epoch 289
Loss = 4.6848e-01, PNorm = 53.9693, GNorm = 44.9260, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.517654
Epoch 290
Validation binary_cross_entropy = 0.507751
Epoch 291
Validation binary_cross_entropy = 0.500375
Epoch 292
Validation binary_cross_entropy = 0.521247
Epoch 293
Validation binary_cross_entropy = 0.534611
Epoch 294
Loss = 4.6120e-01, PNorm = 53.9724, GNorm = 39.7803, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.549081
Epoch 295
Validation binary_cross_entropy = 0.516392
Epoch 296
Validation binary_cross_entropy = 0.470525
Epoch 297
Validation binary_cross_entropy = 0.423944
Epoch 298
Validation binary_cross_entropy = 0.447042
Epoch 299
Loss = 7.7403e-01, PNorm = 53.9754, GNorm = 65.8375, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.517007
Model 0 best validation binary_cross_entropy = 0.338327 on epoch 227
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.358662
Ensemble test binary_cross_entropy = 0.358662
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=300, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.35000000000000003, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 5,506,601
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 66.711739
Epoch 1
Validation binary_cross_entropy = 6.905139
Epoch 2
Validation binary_cross_entropy = 88.773834
Epoch 3
Validation binary_cross_entropy = 11.869384
Epoch 4
Loss = 1.4657e+02, PNorm = 52.2800, GNorm = 2445.1530, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 7.312339
Epoch 5
Validation binary_cross_entropy = 44.624786
Epoch 6
Validation binary_cross_entropy = 41.526089
Epoch 7
Validation binary_cross_entropy = 11.361249
Epoch 8
Validation binary_cross_entropy = 11.780601
Epoch 9
Loss = 6.7418e+01, PNorm = 52.4737, GNorm = 1971.1642, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 37.969391
Epoch 10
Validation binary_cross_entropy = 7.370788
Epoch 11
Validation binary_cross_entropy = 56.962635
Epoch 12
Validation binary_cross_entropy = 7.575702
Epoch 13
Validation binary_cross_entropy = 18.605911
Epoch 14
Loss = 6.5404e+01, PNorm = 52.7116, GNorm = 1932.9600, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 32.093006
Epoch 15
Validation binary_cross_entropy = 6.986669
Epoch 16
Validation binary_cross_entropy = 19.010983
Epoch 17
Validation binary_cross_entropy = 14.003833
Epoch 18
Validation binary_cross_entropy = 14.381049
Epoch 19
Loss = 6.4450e+01, PNorm = 52.9037, GNorm = 1077.8901, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 19.581507
Epoch 20
Validation binary_cross_entropy = 7.573168
Epoch 21
Validation binary_cross_entropy = 15.870440
Epoch 22
Validation binary_cross_entropy = 40.729549
Epoch 23
Validation binary_cross_entropy = 5.968122
Epoch 24
Loss = 3.4928e+01, PNorm = 53.0456, GNorm = 842.0351, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 6.548649
Epoch 25
Validation binary_cross_entropy = 21.373981
Epoch 26
Validation binary_cross_entropy = 13.963014
Epoch 27
Validation binary_cross_entropy = 6.923463
Epoch 28
Validation binary_cross_entropy = 4.982692
Epoch 29
Loss = 3.0246e+01, PNorm = 53.1551, GNorm = 539.3223, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 34.068928
Epoch 30
Validation binary_cross_entropy = 29.657206
Epoch 31
Validation binary_cross_entropy = 7.820316
Epoch 32
Validation binary_cross_entropy = 12.511738
Epoch 33
Validation binary_cross_entropy = 56.289379
Epoch 34
Loss = 3.2739e+01, PNorm = 53.2417, GNorm = 1245.9169, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 3.640546
Epoch 35
Validation binary_cross_entropy = 4.739942
Epoch 36
Validation binary_cross_entropy = 9.033601
Epoch 37
Validation binary_cross_entropy = 12.019307
Epoch 38
Validation binary_cross_entropy = 4.784234
Epoch 39
Loss = 1.8921e+01, PNorm = 53.3153, GNorm = 651.0924, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 5.041221
Epoch 40
Validation binary_cross_entropy = 7.669727
Epoch 41
Validation binary_cross_entropy = 5.238011
Epoch 42
Validation binary_cross_entropy = 4.409301
Epoch 43
Validation binary_cross_entropy = 4.657889
Epoch 44
Loss = 1.4718e+01, PNorm = 53.3733, GNorm = 420.9139, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 17.627943
Epoch 45
Validation binary_cross_entropy = 31.104309
Epoch 46
Validation binary_cross_entropy = 4.218100
Epoch 47
Validation binary_cross_entropy = 3.958368
Epoch 48
Validation binary_cross_entropy = 12.941389
Epoch 49
Loss = 1.7793e+01, PNorm = 53.4219, GNorm = 382.4390, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 23.707212
Epoch 50
Validation binary_cross_entropy = 16.784384
Epoch 51
Validation binary_cross_entropy = 8.219658
Epoch 52
Validation binary_cross_entropy = 4.107077
Epoch 53
Validation binary_cross_entropy = 9.347556
Epoch 54
Loss = 5.1066e+00, PNorm = 53.4674, GNorm = 340.3650, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 6.562132
Epoch 55
Validation binary_cross_entropy = 4.823894
Epoch 56
Validation binary_cross_entropy = 4.810583
Epoch 57
Validation binary_cross_entropy = 4.697666
Epoch 58
Validation binary_cross_entropy = 39.279675
Epoch 59
Loss = 2.3943e+01, PNorm = 53.5078, GNorm = 837.4156, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 19.719984
Epoch 60
Validation binary_cross_entropy = 3.094869
Epoch 61
Validation binary_cross_entropy = 3.507255
Epoch 62
Validation binary_cross_entropy = 4.653048
Epoch 63
Validation binary_cross_entropy = 12.670618
Epoch 64
Loss = 6.3495e+00, PNorm = 53.5458, GNorm = 149.6899, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 10.021787
Epoch 65
Validation binary_cross_entropy = 5.894351
Epoch 66
Validation binary_cross_entropy = 4.798378
Epoch 67
Validation binary_cross_entropy = 4.475395
Epoch 68
Validation binary_cross_entropy = 3.107026
Epoch 69
Loss = 7.4669e+00, PNorm = 53.5794, GNorm = 239.2938, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 2.277888
Epoch 70
Validation binary_cross_entropy = 2.176982
Epoch 71
Validation binary_cross_entropy = 4.791189
Epoch 72
Validation binary_cross_entropy = 9.020576
Epoch 73
Validation binary_cross_entropy = 9.632008
Epoch 74
Loss = 7.7671e+00, PNorm = 53.6120, GNorm = 296.7436, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 1.994082
Epoch 75
Validation binary_cross_entropy = 1.620575
Epoch 76
Validation binary_cross_entropy = 2.262575
Epoch 77
Validation binary_cross_entropy = 15.942932
Epoch 78
Validation binary_cross_entropy = 18.567081
Epoch 79
Loss = 4.5609e+00, PNorm = 53.6447, GNorm = 219.7093, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 10.340472
Epoch 80
Validation binary_cross_entropy = 4.062716
Epoch 81
Validation binary_cross_entropy = 3.478871
Epoch 82
Validation binary_cross_entropy = 6.363689
Epoch 83
Validation binary_cross_entropy = 9.973246
Epoch 84
Loss = 3.1618e+00, PNorm = 53.6722, GNorm = 176.7528, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 7.540684
Epoch 85
Validation binary_cross_entropy = 5.594365
Epoch 86
Validation binary_cross_entropy = 3.695917
Epoch 87
Validation binary_cross_entropy = 2.441327
Epoch 88
Validation binary_cross_entropy = 2.364695
Epoch 89
Loss = 4.0089e+00, PNorm = 53.6961, GNorm = 141.9154, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 3.090140
Epoch 90
Validation binary_cross_entropy = 2.437551
Epoch 91
Validation binary_cross_entropy = 1.137522
Epoch 92
Validation binary_cross_entropy = 1.374970
Epoch 93
Validation binary_cross_entropy = 1.291801
Epoch 94
Loss = 5.7137e+00, PNorm = 53.7183, GNorm = 286.1921, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 1.514937
Epoch 95
Validation binary_cross_entropy = 5.597681
Epoch 96
Validation binary_cross_entropy = 3.531655
Epoch 97
Validation binary_cross_entropy = 0.972856
Epoch 98
Validation binary_cross_entropy = 1.095631
Epoch 99
Loss = 4.1477e+00, PNorm = 53.7404, GNorm = 145.6106, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.989291
Epoch 100
Validation binary_cross_entropy = 1.442948
Epoch 101
Validation binary_cross_entropy = 1.600405
Epoch 102
Validation binary_cross_entropy = 0.873821
Epoch 103
Validation binary_cross_entropy = 1.040111
Epoch 104
Loss = 2.4509e+00, PNorm = 53.7621, GNorm = 241.6344, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 1.162922
Epoch 105
Validation binary_cross_entropy = 3.673018
Epoch 106
Validation binary_cross_entropy = 3.297697
Epoch 107
Validation binary_cross_entropy = 1.397691
Epoch 108
Validation binary_cross_entropy = 0.959419
Epoch 109
Loss = 3.2969e+00, PNorm = 53.7833, GNorm = 130.7991, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.762666
Epoch 110
Validation binary_cross_entropy = 0.734187
Epoch 111
Validation binary_cross_entropy = 0.745208
Epoch 112
Validation binary_cross_entropy = 0.851834
Epoch 113
Validation binary_cross_entropy = 0.976407
Epoch 114
Loss = 1.5623e+00, PNorm = 53.8002, GNorm = 77.1322, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 1.215680
Epoch 115
Validation binary_cross_entropy = 1.180103
Epoch 116
Validation binary_cross_entropy = 1.031141
Epoch 117
Validation binary_cross_entropy = 0.932208
Epoch 118
Validation binary_cross_entropy = 0.980236
Epoch 119
Loss = 2.5508e+00, PNorm = 53.8144, GNorm = 118.0976, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.988839
Epoch 120
Validation binary_cross_entropy = 1.183703
Epoch 121
Validation binary_cross_entropy = 1.699477
Epoch 122
Validation binary_cross_entropy = 1.750952
Epoch 123
Validation binary_cross_entropy = 1.116490
Epoch 124
Loss = 2.1578e+00, PNorm = 53.8273, GNorm = 175.9145, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.799459
Epoch 125
Validation binary_cross_entropy = 0.556282
Epoch 126
Validation binary_cross_entropy = 0.490045
Epoch 127
Validation binary_cross_entropy = 0.536406
Epoch 128
Validation binary_cross_entropy = 0.488356
Epoch 129
Loss = 1.1461e+00, PNorm = 53.8425, GNorm = 90.7683, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.578167
Epoch 130
Validation binary_cross_entropy = 0.743922
Epoch 131
Validation binary_cross_entropy = 0.784650
Epoch 132
Validation binary_cross_entropy = 0.743570
Epoch 133
Validation binary_cross_entropy = 0.796092
Epoch 134
Loss = 2.7756e+00, PNorm = 53.8575, GNorm = 98.9687, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.845037
Epoch 135
Validation binary_cross_entropy = 0.851445
Epoch 136
Validation binary_cross_entropy = 0.881298
Epoch 137
Validation binary_cross_entropy = 0.968394
Epoch 138
Validation binary_cross_entropy = 0.973443
Epoch 139
Loss = 6.6702e-01, PNorm = 53.8705, GNorm = 63.9571, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.835953
Epoch 140
Validation binary_cross_entropy = 0.767352
Epoch 141
Validation binary_cross_entropy = 0.773708
Epoch 142
Validation binary_cross_entropy = 1.260770
Epoch 143
Validation binary_cross_entropy = 1.355752
Epoch 144
Loss = 1.4916e+00, PNorm = 53.8810, GNorm = 102.1522, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 1.042801
Epoch 145
Validation binary_cross_entropy = 0.529899
Epoch 146
Validation binary_cross_entropy = 0.425370
Epoch 147
Validation binary_cross_entropy = 0.541486
Epoch 148
Validation binary_cross_entropy = 0.904805
Epoch 149
Loss = 1.7534e+00, PNorm = 53.8911, GNorm = 124.9779, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 1.161381
Epoch 150
Validation binary_cross_entropy = 1.009113
Epoch 151
Validation binary_cross_entropy = 0.728582
Epoch 152
Validation binary_cross_entropy = 0.516967
Epoch 153
Validation binary_cross_entropy = 0.452389
Epoch 154
Loss = 1.1121e+00, PNorm = 53.9011, GNorm = 66.7817, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.452345
Epoch 155
Validation binary_cross_entropy = 0.504110
Epoch 156
Validation binary_cross_entropy = 0.615565
Epoch 157
Validation binary_cross_entropy = 0.772364
Epoch 158
Validation binary_cross_entropy = 0.946722
Epoch 159
Loss = 1.1386e+00, PNorm = 53.9099, GNorm = 67.8433, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 1.149305
Epoch 160
Validation binary_cross_entropy = 1.068529
Epoch 161
Validation binary_cross_entropy = 0.826155
Epoch 162
Validation binary_cross_entropy = 0.603564
Epoch 163
Validation binary_cross_entropy = 0.508548
Epoch 164
Loss = 1.0640e+00, PNorm = 53.9188, GNorm = 68.7055, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.502014
Epoch 165
Validation binary_cross_entropy = 0.525152
Epoch 166
Validation binary_cross_entropy = 0.508341
Epoch 167
Validation binary_cross_entropy = 0.557332
Epoch 168
Validation binary_cross_entropy = 0.707022
Epoch 169
Loss = 1.4058e+00, PNorm = 53.9273, GNorm = 58.5835, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.626699
Epoch 170
Validation binary_cross_entropy = 0.654235
Epoch 171
Validation binary_cross_entropy = 0.626303
Epoch 172
Validation binary_cross_entropy = 0.643460
Epoch 173
Validation binary_cross_entropy = 0.659083
Epoch 174
Loss = 1.5885e+00, PNorm = 53.9354, GNorm = 112.0822, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.662309
Epoch 175
Validation binary_cross_entropy = 0.841055
Epoch 176
Validation binary_cross_entropy = 0.949950
Epoch 177
Validation binary_cross_entropy = 0.779254
Epoch 178
Validation binary_cross_entropy = 0.762517
Epoch 179
Loss = 9.6598e-01, PNorm = 53.9438, GNorm = 59.0057, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.626467
Epoch 180
Validation binary_cross_entropy = 0.571648
Epoch 181
Validation binary_cross_entropy = 0.593759
Epoch 182
Validation binary_cross_entropy = 0.634353
Epoch 183
Validation binary_cross_entropy = 0.766947
Epoch 184
Loss = 8.1254e-01, PNorm = 53.9520, GNorm = 45.4460, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.899732
Epoch 185
Validation binary_cross_entropy = 0.862218
Epoch 186
Validation binary_cross_entropy = 0.636577
Epoch 187
Validation binary_cross_entropy = 0.458958
Epoch 188
Validation binary_cross_entropy = 0.453108
Epoch 189
Loss = 6.6764e-01, PNorm = 53.9594, GNorm = 77.1126, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.470525
Epoch 190
Validation binary_cross_entropy = 0.516096
Epoch 191
Validation binary_cross_entropy = 0.544058
Epoch 192
Validation binary_cross_entropy = 0.563447
Epoch 193
Validation binary_cross_entropy = 0.555457
Epoch 194
Loss = 8.1219e-01, PNorm = 53.9667, GNorm = 65.0373, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.526894
Epoch 195
Validation binary_cross_entropy = 0.535430
Epoch 196
Validation binary_cross_entropy = 0.591042
Epoch 197
Validation binary_cross_entropy = 0.642622
Epoch 198
Validation binary_cross_entropy = 0.617332
Epoch 199
Loss = 6.1628e-01, PNorm = 53.9738, GNorm = 63.0063, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.559780
Epoch 200
Validation binary_cross_entropy = 0.491840
Epoch 201
Validation binary_cross_entropy = 0.488940
Epoch 202
Validation binary_cross_entropy = 0.554585
Epoch 203
Validation binary_cross_entropy = 0.667093
Epoch 204
Loss = 8.6601e-01, PNorm = 53.9802, GNorm = 90.5965, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.653306
Epoch 205
Validation binary_cross_entropy = 0.569503
Epoch 206
Validation binary_cross_entropy = 0.545068
Epoch 207
Validation binary_cross_entropy = 0.509811
Epoch 208
Validation binary_cross_entropy = 0.515208
Epoch 209
Loss = 5.8133e-01, PNorm = 53.9861, GNorm = 30.0756, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.528144
Epoch 210
Validation binary_cross_entropy = 0.555090
Epoch 211
Validation binary_cross_entropy = 0.603319
Epoch 212
Validation binary_cross_entropy = 0.659689
Epoch 213
Validation binary_cross_entropy = 0.674600
Epoch 214
Loss = 7.4856e-01, PNorm = 53.9913, GNorm = 51.9405, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.621930
Epoch 215
Validation binary_cross_entropy = 0.590391
Epoch 216
Validation binary_cross_entropy = 0.578867
Epoch 217
Validation binary_cross_entropy = 0.559647
Epoch 218
Validation binary_cross_entropy = 0.531003
Epoch 219
Loss = 8.1103e-01, PNorm = 53.9966, GNorm = 76.8124, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.504062
Epoch 220
Validation binary_cross_entropy = 0.486542
Epoch 221
Validation binary_cross_entropy = 0.503637
Epoch 222
Validation binary_cross_entropy = 0.540358
Epoch 223
Validation binary_cross_entropy = 0.559589
Epoch 224
Loss = 9.7451e-01, PNorm = 54.0022, GNorm = 98.0163, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.548417
Epoch 225
Validation binary_cross_entropy = 0.539538
Epoch 226
Validation binary_cross_entropy = 0.514603
Epoch 227
Validation binary_cross_entropy = 0.502248
Epoch 228
Validation binary_cross_entropy = 0.502582
Epoch 229
Loss = 6.9753e-01, PNorm = 54.0076, GNorm = 41.5902, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.519647
Epoch 230
Validation binary_cross_entropy = 0.602998
Epoch 231
Validation binary_cross_entropy = 0.746641
Epoch 232
Validation binary_cross_entropy = 0.631685
Epoch 233
Validation binary_cross_entropy = 0.508077
Epoch 234
Loss = 6.3683e-01, PNorm = 54.0126, GNorm = 57.2254, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.455715
Epoch 235
Validation binary_cross_entropy = 0.444636
Epoch 236
Validation binary_cross_entropy = 0.457747
Epoch 237
Validation binary_cross_entropy = 0.462846
Epoch 238
Validation binary_cross_entropy = 0.486642
Epoch 239
Loss = 5.1246e-01, PNorm = 54.0180, GNorm = 41.3876, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.539267
Epoch 240
Validation binary_cross_entropy = 0.534546
Epoch 241
Validation binary_cross_entropy = 0.525189
Epoch 242
Validation binary_cross_entropy = 0.483168
Epoch 243
Validation binary_cross_entropy = 0.422281
Epoch 244
Loss = 6.3903e-01, PNorm = 54.0234, GNorm = 46.8817, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.381885
Epoch 245
Validation binary_cross_entropy = 0.384275
Epoch 246
Validation binary_cross_entropy = 0.406288
Epoch 247
Validation binary_cross_entropy = 0.435367
Epoch 248
Validation binary_cross_entropy = 0.471576
Epoch 249
Loss = 8.3449e-01, PNorm = 54.0283, GNorm = 40.1535, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.485441
Epoch 250
Validation binary_cross_entropy = 0.493241
Epoch 251
Validation binary_cross_entropy = 0.498607
Epoch 252
Validation binary_cross_entropy = 0.519545
Epoch 253
Validation binary_cross_entropy = 0.542773
Epoch 254
Loss = 7.2831e-01, PNorm = 54.0334, GNorm = 44.5371, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.546717
Epoch 255
Validation binary_cross_entropy = 0.540354
Epoch 256
Validation binary_cross_entropy = 0.523075
Epoch 257
Validation binary_cross_entropy = 0.512218
Epoch 258
Validation binary_cross_entropy = 0.508345
Epoch 259
Loss = 7.8405e-01, PNorm = 54.0378, GNorm = 48.8215, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.494455
Epoch 260
Validation binary_cross_entropy = 0.492704
Epoch 261
Validation binary_cross_entropy = 0.513983
Epoch 262
Validation binary_cross_entropy = 0.515551
Epoch 263
Validation binary_cross_entropy = 0.517409
Epoch 264
Loss = 3.9578e-01, PNorm = 54.0417, GNorm = 42.9065, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.494563
Epoch 265
Validation binary_cross_entropy = 0.457378
Epoch 266
Validation binary_cross_entropy = 0.435614
Epoch 267
Validation binary_cross_entropy = 0.425278
Epoch 268
Validation binary_cross_entropy = 0.422346
Epoch 269
Loss = 6.0199e-01, PNorm = 54.0456, GNorm = 35.4245, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.427607
Epoch 270
Validation binary_cross_entropy = 0.443883
Epoch 271
Validation binary_cross_entropy = 0.451114
Epoch 272
Validation binary_cross_entropy = 0.457351
Epoch 273
Validation binary_cross_entropy = 0.453758
Epoch 274
Loss = 4.6581e-01, PNorm = 54.0497, GNorm = 47.1301, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.462274
Epoch 275
Validation binary_cross_entropy = 0.531746
Epoch 276
Validation binary_cross_entropy = 0.536418
Epoch 277
Validation binary_cross_entropy = 0.494900
Epoch 278
Validation binary_cross_entropy = 0.481747
Epoch 279
Loss = 5.5005e-01, PNorm = 54.0546, GNorm = 46.5442, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.504367
Epoch 280
Validation binary_cross_entropy = 0.522631
Epoch 281
Validation binary_cross_entropy = 0.510166
Epoch 282
Validation binary_cross_entropy = 0.483614
Epoch 283
Validation binary_cross_entropy = 0.464305
Epoch 284
Loss = 5.0321e-01, PNorm = 54.0597, GNorm = 41.5585, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.459764
Epoch 285
Validation binary_cross_entropy = 0.460539
Epoch 286
Validation binary_cross_entropy = 0.475211
Epoch 287
Validation binary_cross_entropy = 0.485665
Epoch 288
Validation binary_cross_entropy = 0.535553
Epoch 289
Loss = 3.8298e-01, PNorm = 54.0646, GNorm = 38.7359, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.556569
Epoch 290
Validation binary_cross_entropy = 0.492378
Epoch 291
Validation binary_cross_entropy = 0.428821
Epoch 292
Validation binary_cross_entropy = 0.386060
Epoch 293
Validation binary_cross_entropy = 0.365740
Epoch 294
Loss = 3.8085e-01, PNorm = 54.0691, GNorm = 42.9653, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.379679
Epoch 295
Validation binary_cross_entropy = 0.443141
Epoch 296
Validation binary_cross_entropy = 0.535688
Epoch 297
Validation binary_cross_entropy = 0.478597
Epoch 298
Validation binary_cross_entropy = 0.415184
Epoch 299
Loss = 4.7260e-01, PNorm = 54.0732, GNorm = 29.9722, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.419391
Model 0 best validation binary_cross_entropy = 0.365740 on epoch 293
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.387466
Ensemble test binary_cross_entropy = 0.387466
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=300, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.35000000000000003, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 5,506,601
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 90.445862
Epoch 1
Validation binary_cross_entropy = 7.996165
Epoch 2
Validation binary_cross_entropy = 68.249702
Epoch 3
Validation binary_cross_entropy = 25.331034
Epoch 4
Loss = 8.0822e+01, PNorm = 52.2776, GNorm = 1402.1555, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 10.699017
Epoch 5
Validation binary_cross_entropy = 29.833979
Epoch 6
Validation binary_cross_entropy = 15.355617
Epoch 7
Validation binary_cross_entropy = 31.111198
Epoch 8
Validation binary_cross_entropy = 21.759710
Epoch 9
Loss = 4.7377e+01, PNorm = 52.4695, GNorm = 698.5620, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 40.401482
Epoch 10
Validation binary_cross_entropy = 10.355879
Epoch 11
Validation binary_cross_entropy = 33.840645
Epoch 12
Validation binary_cross_entropy = 8.970632
Epoch 13
Validation binary_cross_entropy = 80.895233
Epoch 14
Loss = 8.4304e+01, PNorm = 52.7197, GNorm = 1173.8091, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 17.362865
Epoch 15
Validation binary_cross_entropy = 43.386383
Epoch 16
Validation binary_cross_entropy = 32.789623
Epoch 17
Validation binary_cross_entropy = 14.339193
Epoch 18
Validation binary_cross_entropy = 28.610405
Epoch 19
Loss = 5.5453e+01, PNorm = 52.9346, GNorm = 1124.7572, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 7.579597
Epoch 20
Validation binary_cross_entropy = 7.919227
Epoch 21
Validation binary_cross_entropy = 54.756432
Epoch 22
Validation binary_cross_entropy = 6.550836
Epoch 23
Validation binary_cross_entropy = 18.863976
Epoch 24
Loss = 3.9146e+01, PNorm = 53.0944, GNorm = 1146.4438, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 19.152046
Epoch 25
Validation binary_cross_entropy = 10.567916
Epoch 26
Validation binary_cross_entropy = 18.256842
Epoch 27
Validation binary_cross_entropy = 13.390800
Epoch 28
Validation binary_cross_entropy = 12.036621
Epoch 29
Loss = 6.1519e+01, PNorm = 53.2155, GNorm = 948.2271, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 13.718481
Epoch 30
Validation binary_cross_entropy = 12.135377
Epoch 31
Validation binary_cross_entropy = 14.781659
Epoch 32
Validation binary_cross_entropy = 13.674011
Epoch 33
Validation binary_cross_entropy = 17.473570
Epoch 34
Loss = 3.2936e+01, PNorm = 53.3086, GNorm = 980.2163, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 25.087322
Epoch 35
Validation binary_cross_entropy = 19.626823
Epoch 36
Validation binary_cross_entropy = 16.169842
Epoch 37
Validation binary_cross_entropy = 13.892993
Epoch 38
Validation binary_cross_entropy = 15.950139
Epoch 39
Loss = 3.3151e+01, PNorm = 53.3847, GNorm = 496.6190, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 7.525748
Epoch 40
Validation binary_cross_entropy = 6.484417
Epoch 41
Validation binary_cross_entropy = 6.756834
Epoch 42
Validation binary_cross_entropy = 24.498882
Epoch 43
Validation binary_cross_entropy = 17.985157
Epoch 44
Loss = 1.7561e+01, PNorm = 53.4491, GNorm = 412.4541, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 5.793764
Epoch 45
Validation binary_cross_entropy = 15.235202
Epoch 46
Validation binary_cross_entropy = 7.602507
Epoch 47
Validation binary_cross_entropy = 6.384445
Epoch 48
Validation binary_cross_entropy = 22.788443
Epoch 49
Loss = 2.1721e+01, PNorm = 53.5059, GNorm = 360.8964, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 62.350842
Epoch 50
Validation binary_cross_entropy = 45.231083
Epoch 51
Validation binary_cross_entropy = 5.786988
Epoch 52
Validation binary_cross_entropy = 5.933149
Epoch 53
Validation binary_cross_entropy = 22.659643
Epoch 54
Loss = 1.7576e+01, PNorm = 53.5578, GNorm = 574.1582, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 27.161345
Epoch 55
Validation binary_cross_entropy = 10.293283
Epoch 56
Validation binary_cross_entropy = 6.997396
Epoch 57
Validation binary_cross_entropy = 7.341091
Epoch 58
Validation binary_cross_entropy = 9.438984
Epoch 59
Loss = 5.8814e+00, PNorm = 53.6050, GNorm = 246.2426, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 9.504696
Epoch 60
Validation binary_cross_entropy = 17.083746
Epoch 61
Validation binary_cross_entropy = 27.771307
Epoch 62
Validation binary_cross_entropy = 10.784490
Epoch 63
Validation binary_cross_entropy = 3.404907
Epoch 64
Loss = 1.5333e+01, PNorm = 53.6440, GNorm = 596.1736, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 13.399210
Epoch 65
Validation binary_cross_entropy = 38.129826
Epoch 66
Validation binary_cross_entropy = 23.979532
Epoch 67
Validation binary_cross_entropy = 4.106162
Epoch 68
Validation binary_cross_entropy = 4.202935
Epoch 69
Loss = 1.3286e+01, PNorm = 53.6779, GNorm = 350.6968, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 9.487676
Epoch 70
Validation binary_cross_entropy = 26.046076
Epoch 71
Validation binary_cross_entropy = 17.607952
Epoch 72
Validation binary_cross_entropy = 5.013453
Epoch 73
Validation binary_cross_entropy = 1.978626
Epoch 74
Loss = 8.5239e+00, PNorm = 53.7099, GNorm = 342.2379, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 2.776116
Epoch 75
Validation binary_cross_entropy = 9.860337
Epoch 76
Validation binary_cross_entropy = 17.532017
Epoch 77
Validation binary_cross_entropy = 3.855292
Epoch 78
Validation binary_cross_entropy = 1.619924
Epoch 79
Loss = 7.1801e+00, PNorm = 53.7389, GNorm = 367.9659, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 1.817392
Epoch 80
Validation binary_cross_entropy = 2.658913
Epoch 81
Validation binary_cross_entropy = 11.938164
Epoch 82
Validation binary_cross_entropy = 4.533118
Epoch 83
Validation binary_cross_entropy = 2.078926
Epoch 84
Loss = 8.7771e+00, PNorm = 53.7644, GNorm = 261.8844, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 1.592415
Epoch 85
Validation binary_cross_entropy = 3.036574
Epoch 86
Validation binary_cross_entropy = 4.687488
Epoch 87
Validation binary_cross_entropy = 4.153141
Epoch 88
Validation binary_cross_entropy = 2.709939
Epoch 89
Loss = 8.1897e+00, PNorm = 53.7901, GNorm = 186.7555, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 2.088699
Epoch 90
Validation binary_cross_entropy = 2.199780
Epoch 91
Validation binary_cross_entropy = 3.623824
Epoch 92
Validation binary_cross_entropy = 6.938686
Epoch 93
Validation binary_cross_entropy = 5.859436
Epoch 94
Loss = 5.2065e+00, PNorm = 53.8118, GNorm = 136.7875, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 2.986943
Epoch 95
Validation binary_cross_entropy = 2.722950
Epoch 96
Validation binary_cross_entropy = 2.424281
Epoch 97
Validation binary_cross_entropy = 2.270993
Epoch 98
Validation binary_cross_entropy = 2.377496
Epoch 99
Loss = 1.4702e+00, PNorm = 53.8301, GNorm = 145.5760, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 1.231045
Epoch 100
Validation binary_cross_entropy = 0.958466
Epoch 101
Validation binary_cross_entropy = 0.768397
Epoch 102
Validation binary_cross_entropy = 0.712483
Epoch 103
Validation binary_cross_entropy = 0.824246
Epoch 104
Loss = 3.3250e+00, PNorm = 53.8458, GNorm = 203.5940, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.872812
Epoch 105
Validation binary_cross_entropy = 0.854654
Epoch 106
Validation binary_cross_entropy = 1.293077
Epoch 107
Validation binary_cross_entropy = 1.222592
Epoch 108
Validation binary_cross_entropy = 0.925361
Epoch 109
Loss = 2.0926e+00, PNorm = 53.8628, GNorm = 160.3047, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.784754
Epoch 110
Validation binary_cross_entropy = 0.793941
Epoch 111
Validation binary_cross_entropy = 0.983319
Epoch 112
Validation binary_cross_entropy = 1.762332
Epoch 113
Validation binary_cross_entropy = 1.832199
Epoch 114
Loss = 1.3764e+00, PNorm = 53.8788, GNorm = 86.4373, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 1.778451
Epoch 115
Validation binary_cross_entropy = 0.756040
Epoch 116
Validation binary_cross_entropy = 0.652512
Epoch 117
Validation binary_cross_entropy = 0.631074
Epoch 118
Validation binary_cross_entropy = 0.823074
Epoch 119
Loss = 2.2588e+00, PNorm = 53.8941, GNorm = 120.9153, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 1.457371
Epoch 120
Validation binary_cross_entropy = 2.200206
Epoch 121
Validation binary_cross_entropy = 1.251796
Epoch 122
Validation binary_cross_entropy = 0.772565
Epoch 123
Validation binary_cross_entropy = 0.674798
Epoch 124
Loss = 1.6272e+00, PNorm = 53.9099, GNorm = 137.1731, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.647724
Epoch 125
Validation binary_cross_entropy = 0.920358
Epoch 126
Validation binary_cross_entropy = 1.785055
Epoch 127
Validation binary_cross_entropy = 2.069401
Epoch 128
Validation binary_cross_entropy = 1.867145
Epoch 129
Loss = 2.2985e+00, PNorm = 53.9250, GNorm = 154.2050, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.872371
Epoch 130
Validation binary_cross_entropy = 0.690811
Epoch 131
Validation binary_cross_entropy = 0.794119
Epoch 132
Validation binary_cross_entropy = 1.114158
Epoch 133
Validation binary_cross_entropy = 2.493412
Epoch 134
Loss = 2.2507e+00, PNorm = 53.9399, GNorm = 177.2405, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 3.712340
Epoch 135
Validation binary_cross_entropy = 3.770672
Epoch 136
Validation binary_cross_entropy = 2.102302
Epoch 137
Validation binary_cross_entropy = 0.799296
Epoch 138
Validation binary_cross_entropy = 0.534542
Epoch 139
Loss = 2.0508e+00, PNorm = 53.9558, GNorm = 176.1229, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.531432
Epoch 140
Validation binary_cross_entropy = 0.512824
Epoch 141
Validation binary_cross_entropy = 0.721456
Epoch 142
Validation binary_cross_entropy = 1.051691
Epoch 143
Validation binary_cross_entropy = 0.793953
Epoch 144
Loss = 1.7792e+00, PNorm = 53.9717, GNorm = 132.0225, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.555634
Epoch 145
Validation binary_cross_entropy = 0.524178
Epoch 146
Validation binary_cross_entropy = 0.579168
Epoch 147
Validation binary_cross_entropy = 0.726385
Epoch 148
Validation binary_cross_entropy = 0.948656
Epoch 149
Loss = 1.5830e+00, PNorm = 53.9851, GNorm = 125.7846, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 1.080380
Epoch 150
Validation binary_cross_entropy = 1.070162
Epoch 151
Validation binary_cross_entropy = 1.251671
Epoch 152
Validation binary_cross_entropy = 1.042435
Epoch 153
Validation binary_cross_entropy = 0.841106
Epoch 154
Loss = 8.4651e-01, PNorm = 53.9970, GNorm = 81.5552, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.729774
Epoch 155
Validation binary_cross_entropy = 0.620333
Epoch 156
Validation binary_cross_entropy = 0.562189
Epoch 157
Validation binary_cross_entropy = 0.550390
Epoch 158
Validation binary_cross_entropy = 0.563155
Epoch 159
Loss = 1.2690e+00, PNorm = 54.0075, GNorm = 135.0845, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.678338
Epoch 160
Validation binary_cross_entropy = 0.820969
Epoch 161
Validation binary_cross_entropy = 0.964487
Epoch 162
Validation binary_cross_entropy = 0.961874
Epoch 163
Validation binary_cross_entropy = 0.799070
Epoch 164
Loss = 2.5044e+00, PNorm = 54.0169, GNorm = 79.6077, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.606586
Epoch 165
Validation binary_cross_entropy = 0.535635
Epoch 166
Validation binary_cross_entropy = 0.550706
Epoch 167
Validation binary_cross_entropy = 0.559348
Epoch 168
Validation binary_cross_entropy = 0.664315
Epoch 169
Loss = 1.4977e+00, PNorm = 54.0259, GNorm = 75.9808, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.841429
Epoch 170
Validation binary_cross_entropy = 0.958777
Epoch 171
Validation binary_cross_entropy = 0.854905
Epoch 172
Validation binary_cross_entropy = 0.652773
Epoch 173
Validation binary_cross_entropy = 0.508319
Epoch 174
Loss = 1.4375e+00, PNorm = 54.0354, GNorm = 209.8781, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.507999
Epoch 175
Validation binary_cross_entropy = 0.670525
Epoch 176
Validation binary_cross_entropy = 0.834401
Epoch 177
Validation binary_cross_entropy = 0.873698
Epoch 178
Validation binary_cross_entropy = 0.878694
Epoch 179
Loss = 7.7736e-01, PNorm = 54.0464, GNorm = 69.1822, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.741886
Epoch 180
Validation binary_cross_entropy = 0.485917
Epoch 181
Validation binary_cross_entropy = 0.397093
Epoch 182
Validation binary_cross_entropy = 0.393781
Epoch 183
Validation binary_cross_entropy = 0.481665
Epoch 184
Loss = 7.8933e-01, PNorm = 54.0574, GNorm = 111.2704, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.540783
Epoch 185
Validation binary_cross_entropy = 0.638839
Epoch 186
Validation binary_cross_entropy = 0.574949
Epoch 187
Validation binary_cross_entropy = 0.560818
Epoch 188
Validation binary_cross_entropy = 0.526256
Epoch 189
Loss = 1.1793e+00, PNorm = 54.0672, GNorm = 84.4262, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.464511
Epoch 190
Validation binary_cross_entropy = 0.447250
Epoch 191
Validation binary_cross_entropy = 0.455112
Epoch 192
Validation binary_cross_entropy = 0.462898
Epoch 193
Validation binary_cross_entropy = 0.471177
Epoch 194
Loss = 1.3498e+00, PNorm = 54.0757, GNorm = 77.9513, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.480993
Epoch 195
Validation binary_cross_entropy = 0.470480
Epoch 196
Validation binary_cross_entropy = 0.453193
Epoch 197
Validation binary_cross_entropy = 0.465838
Epoch 198
Validation binary_cross_entropy = 0.535879
Epoch 199
Loss = 1.0822e+00, PNorm = 54.0834, GNorm = 129.9658, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.519663
Epoch 200
Validation binary_cross_entropy = 0.466245
Epoch 201
Validation binary_cross_entropy = 0.443847
Epoch 202
Validation binary_cross_entropy = 0.396939
Epoch 203
Validation binary_cross_entropy = 0.385627
Epoch 204
Loss = 8.3685e-01, PNorm = 54.0903, GNorm = 88.5549, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.379222
Epoch 205
Validation binary_cross_entropy = 0.402349
Epoch 206
Validation binary_cross_entropy = 0.481629
Epoch 207
Validation binary_cross_entropy = 0.614686
Epoch 208
Validation binary_cross_entropy = 0.548586
Epoch 209
Loss = 4.5204e-01, PNorm = 54.0972, GNorm = 94.7185, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.429841
Epoch 210
Validation binary_cross_entropy = 0.372994
Epoch 211
Validation binary_cross_entropy = 0.363898
Epoch 212
Validation binary_cross_entropy = 0.387631
Epoch 213
Validation binary_cross_entropy = 0.466832
Epoch 214
Loss = 7.8587e-01, PNorm = 54.1037, GNorm = 92.4048, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.554359
Epoch 215
Validation binary_cross_entropy = 0.544032
Epoch 216
Validation binary_cross_entropy = 0.475830
Epoch 217
Validation binary_cross_entropy = 0.426703
Epoch 218
Validation binary_cross_entropy = 0.409720
Epoch 219
Loss = 5.0294e-01, PNorm = 54.1099, GNorm = 66.0311, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.409596
Epoch 220
Validation binary_cross_entropy = 0.428173
Epoch 221
Validation binary_cross_entropy = 0.482127
Epoch 222
Validation binary_cross_entropy = 0.521714
Epoch 223
Validation binary_cross_entropy = 0.552492
Epoch 224
Loss = 7.5917e-01, PNorm = 54.1156, GNorm = 89.0131, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.557512
Epoch 225
Validation binary_cross_entropy = 0.546074
Epoch 226
Validation binary_cross_entropy = 0.547592
Epoch 227
Validation binary_cross_entropy = 0.513835
Epoch 228
Validation binary_cross_entropy = 0.467634
Epoch 229
Loss = 7.0233e-01, PNorm = 54.1213, GNorm = 66.4659, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.425372
Epoch 230
Validation binary_cross_entropy = 0.401079
Epoch 231
Validation binary_cross_entropy = 0.414462
Epoch 232
Validation binary_cross_entropy = 0.492484
Epoch 233
Validation binary_cross_entropy = 0.596382
Epoch 234
Loss = 5.5678e-01, PNorm = 54.1270, GNorm = 40.2636, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.626049
Epoch 235
Validation binary_cross_entropy = 0.580782
Epoch 236
Validation binary_cross_entropy = 0.509011
Epoch 237
Validation binary_cross_entropy = 0.483919
Epoch 238
Validation binary_cross_entropy = 0.494878
Epoch 239
Loss = 6.6237e-01, PNorm = 54.1329, GNorm = 74.0782, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.521793
Epoch 240
Validation binary_cross_entropy = 0.577403
Epoch 241
Validation binary_cross_entropy = 0.619839
Epoch 242
Validation binary_cross_entropy = 0.637859
Epoch 243
Validation binary_cross_entropy = 0.604421
Epoch 244
Loss = 6.4109e-01, PNorm = 54.1383, GNorm = 43.6020, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.572786
Epoch 245
Validation binary_cross_entropy = 0.526776
Epoch 246
Validation binary_cross_entropy = 0.494831
Epoch 247
Validation binary_cross_entropy = 0.476971
Epoch 248
Validation binary_cross_entropy = 0.502195
Epoch 249
Loss = 6.4690e-01, PNorm = 54.1430, GNorm = 80.4759, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.542906
Epoch 250
Validation binary_cross_entropy = 0.600231
Epoch 251
Validation binary_cross_entropy = 0.685675
Epoch 252
Validation binary_cross_entropy = 0.784665
Epoch 253
Validation binary_cross_entropy = 0.827209
Epoch 254
Loss = 6.6951e-01, PNorm = 54.1475, GNorm = 64.9785, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.775240
Epoch 255
Validation binary_cross_entropy = 0.635732
Epoch 256
Validation binary_cross_entropy = 0.535273
Epoch 257
Validation binary_cross_entropy = 0.497462
Epoch 258
Validation binary_cross_entropy = 0.498230
Epoch 259
Loss = 5.4104e-01, PNorm = 54.1514, GNorm = 70.2187, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.516822
Epoch 260
Validation binary_cross_entropy = 0.532782
Epoch 261
Validation binary_cross_entropy = 0.535370
Epoch 262
Validation binary_cross_entropy = 0.563637
Epoch 263
Validation binary_cross_entropy = 0.558882
Epoch 264
Loss = 7.0432e-01, PNorm = 54.1553, GNorm = 58.5453, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.496714
Epoch 265
Validation binary_cross_entropy = 0.495066
Epoch 266
Validation binary_cross_entropy = 0.506450
Epoch 267
Validation binary_cross_entropy = 0.530967
Epoch 268
Validation binary_cross_entropy = 0.598193
Epoch 269
Loss = 5.0719e-01, PNorm = 54.1593, GNorm = 38.6396, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.643798
Epoch 270
Validation binary_cross_entropy = 0.646693
Epoch 271
Validation binary_cross_entropy = 0.597699
Epoch 272
Validation binary_cross_entropy = 0.531248
Epoch 273
Validation binary_cross_entropy = 0.479971
Epoch 274
Loss = 4.8301e-01, PNorm = 54.1629, GNorm = 74.9846, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.462545
Epoch 275
Validation binary_cross_entropy = 0.476420
Epoch 276
Validation binary_cross_entropy = 0.494817
Epoch 277
Validation binary_cross_entropy = 0.514590
Epoch 278
Validation binary_cross_entropy = 0.535567
Epoch 279
Loss = 5.8408e-01, PNorm = 54.1664, GNorm = 65.8830, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.546040
Epoch 280
Validation binary_cross_entropy = 0.537234
Epoch 281
Validation binary_cross_entropy = 0.524344
Epoch 282
Validation binary_cross_entropy = 0.534531
Epoch 283
Validation binary_cross_entropy = 0.534738
Epoch 284
Loss = 7.0949e-01, PNorm = 54.1699, GNorm = 43.5572, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.508644
Epoch 285
Validation binary_cross_entropy = 0.469769
Epoch 286
Validation binary_cross_entropy = 0.441261
Epoch 287
Validation binary_cross_entropy = 0.438895
Epoch 288
Validation binary_cross_entropy = 0.452750
Epoch 289
Loss = 5.7589e-01, PNorm = 54.1737, GNorm = 96.6538, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.475905
Epoch 290
Validation binary_cross_entropy = 0.502785
Epoch 291
Validation binary_cross_entropy = 0.534989
Epoch 292
Validation binary_cross_entropy = 0.549868
Epoch 293
Validation binary_cross_entropy = 0.552237
Epoch 294
Loss = 6.0496e-01, PNorm = 54.1776, GNorm = 76.8669, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.535764
Epoch 295
Validation binary_cross_entropy = 0.521972
Epoch 296
Validation binary_cross_entropy = 0.519303
Epoch 297
Validation binary_cross_entropy = 0.504756
Epoch 298
Validation binary_cross_entropy = 0.503026
Epoch 299
Loss = 5.2941e-01, PNorm = 54.1813, GNorm = 46.5073, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.502096
Model 0 best validation binary_cross_entropy = 0.363898 on epoch 211
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.420375
Ensemble test binary_cross_entropy = 0.420375
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=300, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.35000000000000003, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 5,506,601
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 81.592880
Epoch 1
Validation binary_cross_entropy = 6.251846
Epoch 2
Validation binary_cross_entropy = 76.818733
Epoch 3
Validation binary_cross_entropy = 7.594869
Epoch 4
Loss = 1.6770e+02, PNorm = 52.2796, GNorm = 2324.6227, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 12.548900
Epoch 5
Validation binary_cross_entropy = 61.563553
Epoch 6
Validation binary_cross_entropy = 16.081551
Epoch 7
Validation binary_cross_entropy = 8.061854
Epoch 8
Validation binary_cross_entropy = 47.852901
Epoch 9
Loss = 7.7697e+01, PNorm = 52.4771, GNorm = 2781.1974, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 8.957202
Epoch 10
Validation binary_cross_entropy = 7.459723
Epoch 11
Validation binary_cross_entropy = 70.472679
Epoch 12
Validation binary_cross_entropy = 22.154800
Epoch 13
Validation binary_cross_entropy = 5.843579
Epoch 14
Loss = 5.6948e+01, PNorm = 52.7130, GNorm = 668.3155, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 29.843287
Epoch 15
Validation binary_cross_entropy = 18.654610
Epoch 16
Validation binary_cross_entropy = 11.809379
Epoch 17
Validation binary_cross_entropy = 23.606646
Epoch 18
Validation binary_cross_entropy = 13.137658
Epoch 19
Loss = 5.5348e+01, PNorm = 52.8918, GNorm = 738.5971, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 7.579088
Epoch 20
Validation binary_cross_entropy = 27.807173
Epoch 21
Validation binary_cross_entropy = 29.401525
Epoch 22
Validation binary_cross_entropy = 11.376156
Epoch 23
Validation binary_cross_entropy = 13.292878
Epoch 24
Loss = 1.4245e+01, PNorm = 53.0218, GNorm = 401.5076, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 6.058177
Epoch 25
Validation binary_cross_entropy = 18.046169
Epoch 26
Validation binary_cross_entropy = 16.231760
Epoch 27
Validation binary_cross_entropy = 4.866790
Epoch 28
Validation binary_cross_entropy = 22.823967
Epoch 29
Loss = 2.7604e+01, PNorm = 53.1203, GNorm = 595.1633, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 26.624701
Epoch 30
Validation binary_cross_entropy = 6.998081
Epoch 31
Validation binary_cross_entropy = 6.179551
Epoch 32
Validation binary_cross_entropy = 37.945068
Epoch 33
Validation binary_cross_entropy = 7.290079
Epoch 34
Loss = 2.9116e+01, PNorm = 53.2070, GNorm = 862.8422, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 5.214136
Epoch 35
Validation binary_cross_entropy = 6.019007
Epoch 36
Validation binary_cross_entropy = 33.281593
Epoch 37
Validation binary_cross_entropy = 11.969227
Epoch 38
Validation binary_cross_entropy = 4.920912
Epoch 39
Loss = 2.3071e+01, PNorm = 53.2829, GNorm = 618.7498, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 4.638587
Epoch 40
Validation binary_cross_entropy = 8.266579
Epoch 41
Validation binary_cross_entropy = 26.441204
Epoch 42
Validation binary_cross_entropy = 8.506248
Epoch 43
Validation binary_cross_entropy = 3.920622
Epoch 44
Loss = 1.9769e+01, PNorm = 53.3486, GNorm = 460.7923, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 15.415505
Epoch 45
Validation binary_cross_entropy = 40.749657
Epoch 46
Validation binary_cross_entropy = 13.345400
Epoch 47
Validation binary_cross_entropy = 3.722815
Epoch 48
Validation binary_cross_entropy = 3.865830
Epoch 49
Loss = 1.3616e+01, PNorm = 53.4042, GNorm = 319.8730, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 5.605969
Epoch 50
Validation binary_cross_entropy = 13.328357
Epoch 51
Validation binary_cross_entropy = 7.465192
Epoch 52
Validation binary_cross_entropy = 4.531812
Epoch 53
Validation binary_cross_entropy = 6.009345
Epoch 54
Loss = 7.6008e+00, PNorm = 53.4522, GNorm = 312.5679, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 5.897286
Epoch 55
Validation binary_cross_entropy = 7.737875
Epoch 56
Validation binary_cross_entropy = 12.530874
Epoch 57
Validation binary_cross_entropy = 12.557484
Epoch 58
Validation binary_cross_entropy = 5.422464
Epoch 59
Loss = 8.6045e+00, PNorm = 53.4958, GNorm = 351.8551, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 5.864795
Epoch 60
Validation binary_cross_entropy = 5.222939
Epoch 61
Validation binary_cross_entropy = 4.824473
Epoch 62
Validation binary_cross_entropy = 8.305807
Epoch 63
Validation binary_cross_entropy = 11.292408
Epoch 64
Loss = 1.1695e+01, PNorm = 53.5335, GNorm = 424.4080, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 9.031960
Epoch 65
Validation binary_cross_entropy = 3.682885
Epoch 66
Validation binary_cross_entropy = 3.803397
Epoch 67
Validation binary_cross_entropy = 10.053300
Epoch 68
Validation binary_cross_entropy = 15.029366
Epoch 69
Loss = 5.2514e+00, PNorm = 53.5671, GNorm = 251.5532, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 7.349076
Epoch 70
Validation binary_cross_entropy = 6.395084
Epoch 71
Validation binary_cross_entropy = 2.674321
Epoch 72
Validation binary_cross_entropy = 2.224439
Epoch 73
Validation binary_cross_entropy = 2.626637
Epoch 74
Loss = 3.7994e+00, PNorm = 53.6000, GNorm = 170.6638, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 3.309541
Epoch 75
Validation binary_cross_entropy = 11.772041
Epoch 76
Validation binary_cross_entropy = 3.650560
Epoch 77
Validation binary_cross_entropy = 1.479122
Epoch 78
Validation binary_cross_entropy = 1.478329
Epoch 79
Loss = 4.2134e+00, PNorm = 53.6306, GNorm = 158.3870, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 5.678179
Epoch 80
Validation binary_cross_entropy = 6.883534
Epoch 81
Validation binary_cross_entropy = 2.540673
Epoch 82
Validation binary_cross_entropy = 2.460208
Epoch 83
Validation binary_cross_entropy = 2.349333
Epoch 84
Loss = 3.8018e+00, PNorm = 53.6597, GNorm = 154.1385, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 2.385687
Epoch 85
Validation binary_cross_entropy = 3.433016
Epoch 86
Validation binary_cross_entropy = 4.005698
Epoch 87
Validation binary_cross_entropy = 2.235662
Epoch 88
Validation binary_cross_entropy = 0.975408
Epoch 89
Loss = 3.0338e+00, PNorm = 53.6856, GNorm = 152.3988, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.958514
Epoch 90
Validation binary_cross_entropy = 1.167585
Epoch 91
Validation binary_cross_entropy = 3.550855
Epoch 92
Validation binary_cross_entropy = 2.151670
Epoch 93
Validation binary_cross_entropy = 0.861570
Epoch 94
Loss = 4.8436e+00, PNorm = 53.7081, GNorm = 282.9342, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.788762
Epoch 95
Validation binary_cross_entropy = 0.733326
Epoch 96
Validation binary_cross_entropy = 0.842462
Epoch 97
Validation binary_cross_entropy = 1.014232
Epoch 98
Validation binary_cross_entropy = 0.934360
Epoch 99
Loss = 2.2574e+00, PNorm = 53.7293, GNorm = 139.3413, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.823871
Epoch 100
Validation binary_cross_entropy = 0.764851
Epoch 101
Validation binary_cross_entropy = 0.811529
Epoch 102
Validation binary_cross_entropy = 1.283538
Epoch 103
Validation binary_cross_entropy = 1.188246
Epoch 104
Loss = 2.9268e+00, PNorm = 53.7479, GNorm = 100.6493, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.852032
Epoch 105
Validation binary_cross_entropy = 0.679650
Epoch 106
Validation binary_cross_entropy = 0.635490
Epoch 107
Validation binary_cross_entropy = 0.740970
Epoch 108
Validation binary_cross_entropy = 0.831933
Epoch 109
Loss = 2.8599e+00, PNorm = 53.7649, GNorm = 282.3267, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.709311
Epoch 110
Validation binary_cross_entropy = 0.524726
Epoch 111
Validation binary_cross_entropy = 0.546152
Epoch 112
Validation binary_cross_entropy = 0.567911
Epoch 113
Validation binary_cross_entropy = 0.499108
Epoch 114
Loss = 1.9090e+00, PNorm = 53.7800, GNorm = 87.7996, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.558889
Epoch 115
Validation binary_cross_entropy = 1.372421
Epoch 116
Validation binary_cross_entropy = 1.256902
Epoch 117
Validation binary_cross_entropy = 0.434779
Epoch 118
Validation binary_cross_entropy = 0.396494
Epoch 119
Loss = 1.4145e+00, PNorm = 53.7936, GNorm = 100.4645, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.457765
Epoch 120
Validation binary_cross_entropy = 0.428024
Epoch 121
Validation binary_cross_entropy = 1.402295
Epoch 122
Validation binary_cross_entropy = 0.849906
Epoch 123
Validation binary_cross_entropy = 0.483971
Epoch 124
Loss = 1.7491e+00, PNorm = 53.8073, GNorm = 169.4613, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.644419
Epoch 125
Validation binary_cross_entropy = 0.556735
Epoch 126
Validation binary_cross_entropy = 0.572430
Epoch 127
Validation binary_cross_entropy = 0.737920
Epoch 128
Validation binary_cross_entropy = 0.551046
Epoch 129
Loss = 1.6261e+00, PNorm = 53.8230, GNorm = 91.8083, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.421728
Epoch 130
Validation binary_cross_entropy = 0.416167
Epoch 131
Validation binary_cross_entropy = 0.408401
Epoch 132
Validation binary_cross_entropy = 0.367410
Epoch 133
Validation binary_cross_entropy = 0.350358
Epoch 134
Loss = 1.3194e+00, PNorm = 53.8371, GNorm = 85.1696, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.429303
Epoch 135
Validation binary_cross_entropy = 0.454087
Epoch 136
Validation binary_cross_entropy = 0.394634
Epoch 137
Validation binary_cross_entropy = 0.399466
Epoch 138
Validation binary_cross_entropy = 0.435602
Epoch 139
Loss = 1.2080e+00, PNorm = 53.8497, GNorm = 54.9538, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.468730
Epoch 140
Validation binary_cross_entropy = 0.525282
Epoch 141
Validation binary_cross_entropy = 0.541588
Epoch 142
Validation binary_cross_entropy = 0.547370
Epoch 143
Validation binary_cross_entropy = 0.540246
Epoch 144
Loss = 6.3582e-01, PNorm = 53.8609, GNorm = 38.7626, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.533933
Epoch 145
Validation binary_cross_entropy = 0.551237
Epoch 146
Validation binary_cross_entropy = 0.713994
Epoch 147
Validation binary_cross_entropy = 0.957852
Epoch 148
Validation binary_cross_entropy = 1.048762
Epoch 149
Loss = 1.0297e+00, PNorm = 53.8710, GNorm = 129.4168, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.851368
Epoch 150
Validation binary_cross_entropy = 0.591555
Epoch 151
Validation binary_cross_entropy = 0.442318
Epoch 152
Validation binary_cross_entropy = 0.427582
Epoch 153
Validation binary_cross_entropy = 0.499534
Epoch 154
Loss = 1.1652e+00, PNorm = 53.8813, GNorm = 52.9153, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.682949
Epoch 155
Validation binary_cross_entropy = 0.733136
Epoch 156
Validation binary_cross_entropy = 0.705817
Epoch 157
Validation binary_cross_entropy = 0.581407
Epoch 158
Validation binary_cross_entropy = 0.454250
Epoch 159
Loss = 7.7881e-01, PNorm = 53.8922, GNorm = 59.8572, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.481941
Epoch 160
Validation binary_cross_entropy = 0.559191
Epoch 161
Validation binary_cross_entropy = 0.703816
Epoch 162
Validation binary_cross_entropy = 0.813048
Epoch 163
Validation binary_cross_entropy = 0.688104
Epoch 164
Loss = 1.1393e+00, PNorm = 53.9028, GNorm = 81.6342, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.526493
Epoch 165
Validation binary_cross_entropy = 0.435254
Epoch 166
Validation binary_cross_entropy = 0.379911
Epoch 167
Validation binary_cross_entropy = 0.424034
Epoch 168
Validation binary_cross_entropy = 0.529574
Epoch 169
Loss = 1.2759e+00, PNorm = 53.9118, GNorm = 75.7483, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.432475
Epoch 170
Validation binary_cross_entropy = 0.359167
Epoch 171
Validation binary_cross_entropy = 0.362205
Epoch 172
Validation binary_cross_entropy = 0.401643
Epoch 173
Validation binary_cross_entropy = 0.612282
Epoch 174
Loss = 8.7060e-01, PNorm = 53.9210, GNorm = 87.5739, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.767176
Epoch 175
Validation binary_cross_entropy = 0.480762
Epoch 176
Validation binary_cross_entropy = 0.350630
Epoch 177
Validation binary_cross_entropy = 0.369477
Epoch 178
Validation binary_cross_entropy = 0.434293
Epoch 179
Loss = 8.5531e-01, PNorm = 53.9306, GNorm = 85.5933, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.466589
Epoch 180
Validation binary_cross_entropy = 0.470766
Epoch 181
Validation binary_cross_entropy = 0.458892
Epoch 182
Validation binary_cross_entropy = 0.546036
Epoch 183
Validation binary_cross_entropy = 0.740124
Epoch 184
Loss = 1.0185e+00, PNorm = 53.9404, GNorm = 47.5195, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.843409
Epoch 185
Validation binary_cross_entropy = 0.559943
Epoch 186
Validation binary_cross_entropy = 0.419227
Epoch 187
Validation binary_cross_entropy = 0.376624
Epoch 188
Validation binary_cross_entropy = 0.427913
Epoch 189
Loss = 7.4386e-01, PNorm = 53.9493, GNorm = 51.1310, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.480689
Epoch 190
Validation binary_cross_entropy = 0.492146
Epoch 191
Validation binary_cross_entropy = 0.501183
Epoch 192
Validation binary_cross_entropy = 0.475004
Epoch 193
Validation binary_cross_entropy = 0.425709
Epoch 194
Loss = 1.0965e+00, PNorm = 53.9568, GNorm = 71.1379, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.377450
Epoch 195
Validation binary_cross_entropy = 0.364152
Epoch 196
Validation binary_cross_entropy = 0.373862
Epoch 197
Validation binary_cross_entropy = 0.386050
Epoch 198
Validation binary_cross_entropy = 0.394380
Epoch 199
Loss = 9.6768e-01, PNorm = 53.9636, GNorm = 49.1123, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.393402
Epoch 200
Validation binary_cross_entropy = 0.383385
Epoch 201
Validation binary_cross_entropy = 0.412002
Epoch 202
Validation binary_cross_entropy = 0.434659
Epoch 203
Validation binary_cross_entropy = 0.439055
Epoch 204
Loss = 6.5023e-01, PNorm = 53.9703, GNorm = 62.7819, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.479491
Epoch 205
Validation binary_cross_entropy = 0.574628
Epoch 206
Validation binary_cross_entropy = 0.603351
Epoch 207
Validation binary_cross_entropy = 0.616154
Epoch 208
Validation binary_cross_entropy = 0.559450
Epoch 209
Loss = 6.4084e-01, PNorm = 53.9764, GNorm = 59.0793, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.518394
Epoch 210
Validation binary_cross_entropy = 0.493663
Epoch 211
Validation binary_cross_entropy = 0.486607
Epoch 212
Validation binary_cross_entropy = 0.464429
Epoch 213
Validation binary_cross_entropy = 0.438224
Epoch 214
Loss = 8.9321e-01, PNorm = 53.9815, GNorm = 80.4242, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.423749
Epoch 215
Validation binary_cross_entropy = 0.413956
Epoch 216
Validation binary_cross_entropy = 0.430440
Epoch 217
Validation binary_cross_entropy = 0.434979
Epoch 218
Validation binary_cross_entropy = 0.441210
Epoch 219
Loss = 7.4352e-01, PNorm = 53.9864, GNorm = 72.6529, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.473278
Epoch 220
Validation binary_cross_entropy = 0.525679
Epoch 221
Validation binary_cross_entropy = 0.565488
Epoch 222
Validation binary_cross_entropy = 0.471505
Epoch 223
Validation binary_cross_entropy = 0.417360
Epoch 224
Loss = 5.4986e-01, PNorm = 53.9914, GNorm = 39.8632, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.400159
Epoch 225
Validation binary_cross_entropy = 0.396477
Epoch 226
Validation binary_cross_entropy = 0.396174
Epoch 227
Validation binary_cross_entropy = 0.396699
Epoch 228
Validation binary_cross_entropy = 0.392069
Epoch 229
Loss = 6.9132e-01, PNorm = 53.9964, GNorm = 45.0641, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.395082
Epoch 230
Validation binary_cross_entropy = 0.410913
Epoch 231
Validation binary_cross_entropy = 0.406964
Epoch 232
Validation binary_cross_entropy = 0.395819
Epoch 233
Validation binary_cross_entropy = 0.402039
Epoch 234
Loss = 8.2250e-01, PNorm = 54.0021, GNorm = 63.8060, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.405825
Epoch 235
Validation binary_cross_entropy = 0.392758
Epoch 236
Validation binary_cross_entropy = 0.394872
Epoch 237
Validation binary_cross_entropy = 0.399568
Epoch 238
Validation binary_cross_entropy = 0.402776
Epoch 239
Loss = 6.5890e-01, PNorm = 54.0081, GNorm = 48.0802, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.463816
Epoch 240
Validation binary_cross_entropy = 0.645653
Epoch 241
Validation binary_cross_entropy = 0.729958
Epoch 242
Validation binary_cross_entropy = 0.517408
Epoch 243
Validation binary_cross_entropy = 0.379612
Epoch 244
Loss = 6.9291e-01, PNorm = 54.0148, GNorm = 77.8262, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.358268
Epoch 245
Validation binary_cross_entropy = 0.375830
Epoch 246
Validation binary_cross_entropy = 0.411861
Epoch 247
Validation binary_cross_entropy = 0.419198
Epoch 248
Validation binary_cross_entropy = 0.427211
Epoch 249
Loss = 4.5186e-01, PNorm = 54.0211, GNorm = 32.9316, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.457623
Epoch 250
Validation binary_cross_entropy = 0.486419
Epoch 251
Validation binary_cross_entropy = 0.472486
Epoch 252
Validation binary_cross_entropy = 0.431811
Epoch 253
Validation binary_cross_entropy = 0.433030
Epoch 254
Loss = 4.7347e-01, PNorm = 54.0259, GNorm = 30.0383, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.457705
Epoch 255
Validation binary_cross_entropy = 0.493621
Epoch 256
Validation binary_cross_entropy = 0.510780
Epoch 257
Validation binary_cross_entropy = 0.516217
Epoch 258
Validation binary_cross_entropy = 0.494841
Epoch 259
Loss = 5.5417e-01, PNorm = 54.0296, GNorm = 35.9161, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.463960
Epoch 260
Validation binary_cross_entropy = 0.442118
Epoch 261
Validation binary_cross_entropy = 0.421816
Epoch 262
Validation binary_cross_entropy = 0.406593
Epoch 263
Validation binary_cross_entropy = 0.414781
Epoch 264
Loss = 6.4187e-01, PNorm = 54.0332, GNorm = 54.7275, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.427334
Epoch 265
Validation binary_cross_entropy = 0.458300
Epoch 266
Validation binary_cross_entropy = 0.486135
Epoch 267
Validation binary_cross_entropy = 0.493200
Epoch 268
Validation binary_cross_entropy = 0.440713
Epoch 269
Loss = 5.2226e-01, PNorm = 54.0374, GNorm = 49.9550, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.407897
Epoch 270
Validation binary_cross_entropy = 0.391200
Epoch 271
Validation binary_cross_entropy = 0.395937
Epoch 272
Validation binary_cross_entropy = 0.421521
Epoch 273
Validation binary_cross_entropy = 0.452223
Epoch 274
Loss = 8.2395e-01, PNorm = 54.0415, GNorm = 61.4592, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.464398
Epoch 275
Validation binary_cross_entropy = 0.465321
Epoch 276
Validation binary_cross_entropy = 0.478349
Epoch 277
Validation binary_cross_entropy = 0.520088
Epoch 278
Validation binary_cross_entropy = 0.527773
Epoch 279
Loss = 7.7688e-01, PNorm = 54.0453, GNorm = 41.0379, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.486198
Epoch 280
Validation binary_cross_entropy = 0.455052
Epoch 281
Validation binary_cross_entropy = 0.426257
Epoch 282
Validation binary_cross_entropy = 0.427153
Epoch 283
Validation binary_cross_entropy = 0.424015
Epoch 284
Loss = 6.5681e-01, PNorm = 54.0495, GNorm = 50.4554, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.447064
Epoch 285
Validation binary_cross_entropy = 0.502054
Epoch 286
Validation binary_cross_entropy = 0.466416
Epoch 287
Validation binary_cross_entropy = 0.432670
Epoch 288
Validation binary_cross_entropy = 0.420945
Epoch 289
Loss = 4.9963e-01, PNorm = 54.0535, GNorm = 29.7294, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.412187
Epoch 290
Validation binary_cross_entropy = 0.414279
Epoch 291
Validation binary_cross_entropy = 0.423470
Epoch 292
Validation binary_cross_entropy = 0.430580
Epoch 293
Validation binary_cross_entropy = 0.417809
Epoch 294
Loss = 5.4665e-01, PNorm = 54.0574, GNorm = 47.7599, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.417839
Epoch 295
Validation binary_cross_entropy = 0.428398
Epoch 296
Validation binary_cross_entropy = 0.437936
Epoch 297
Validation binary_cross_entropy = 0.446867
Epoch 298
Validation binary_cross_entropy = 0.459513
Epoch 299
Loss = 4.8408e-01, PNorm = 54.0610, GNorm = 40.9603, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.467204
Model 0 best validation binary_cross_entropy = 0.350358 on epoch 133
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.713911
Ensemble test binary_cross_entropy = 0.713911
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=300, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.35000000000000003, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 5,506,601
Moving model to cuda
Epoch 0
Warning: Found a task with predictions all 0s or all 1s
Validation binary_cross_entropy = nan
Epoch 1
Validation binary_cross_entropy = 7.646279
Epoch 2
Validation binary_cross_entropy = 47.571148
Epoch 3
Validation binary_cross_entropy = 20.263090
Epoch 4
Loss = 1.2170e+02, PNorm = 52.2800, GNorm = 1232.9303, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 9.554140
Epoch 5
Validation binary_cross_entropy = 12.051006
Epoch 6
Validation binary_cross_entropy = 77.982658
Epoch 7
Validation binary_cross_entropy = 10.541324
Epoch 8
Validation binary_cross_entropy = 7.603747
Epoch 9
Loss = 1.0128e+02, PNorm = 52.4744, GNorm = 1191.0584, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 85.350319
Epoch 10
Validation binary_cross_entropy = 81.751732
Epoch 11
Validation binary_cross_entropy = 8.148816
Epoch 12
Validation binary_cross_entropy = 8.219574
Epoch 13
Validation binary_cross_entropy = 53.835564
Epoch 14
Loss = 7.4642e+01, PNorm = 52.7066, GNorm = 1317.0798, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 71.344872
Epoch 15
Validation binary_cross_entropy = 8.277628
Epoch 16
Validation binary_cross_entropy = 8.259922
Epoch 17
Validation binary_cross_entropy = 9.018687
Epoch 18
Validation binary_cross_entropy = 37.398464
Epoch 19
Loss = 4.0008e+01, PNorm = 52.8882, GNorm = 558.6324, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 22.087719
Epoch 20
Validation binary_cross_entropy = 6.801594
Epoch 21
Validation binary_cross_entropy = 7.080643
Epoch 22
Validation binary_cross_entropy = 7.983138
Epoch 23
Validation binary_cross_entropy = 14.391944
Epoch 24
Loss = 2.0549e+01, PNorm = 53.0101, GNorm = 551.6638, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 10.148776
Epoch 25
Validation binary_cross_entropy = 7.257642
Epoch 26
Validation binary_cross_entropy = 5.077261
Epoch 27
Validation binary_cross_entropy = 4.671803
Epoch 28
Validation binary_cross_entropy = 4.252287
Epoch 29
Loss = 1.1421e+01, PNorm = 53.0901, GNorm = 339.2312, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 17.464369
Epoch 30
Validation binary_cross_entropy = 7.087685
Epoch 31
Validation binary_cross_entropy = 3.645598
Epoch 32
Validation binary_cross_entropy = 3.281841
Epoch 33
Validation binary_cross_entropy = 4.244312
Epoch 34
Loss = 5.1389e+00, PNorm = 53.1524, GNorm = 139.6337, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 5.032653
Epoch 35
Validation binary_cross_entropy = 5.358254
Epoch 36
Validation binary_cross_entropy = 8.053830
Epoch 37
Validation binary_cross_entropy = 6.035152
Epoch 38
Validation binary_cross_entropy = 4.140841
Epoch 39
Loss = 1.5804e+01, PNorm = 53.2059, GNorm = 346.0970, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 4.377496
Epoch 40
Validation binary_cross_entropy = 11.092529
Epoch 41
Validation binary_cross_entropy = 15.662492
Epoch 42
Validation binary_cross_entropy = 4.500075
Epoch 43
Validation binary_cross_entropy = 8.786421
Epoch 44
Loss = 1.1440e+01, PNorm = 53.2541, GNorm = 408.3859, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 9.365893
Epoch 45
Validation binary_cross_entropy = 8.060320
Epoch 46
Validation binary_cross_entropy = 15.058667
Epoch 47
Validation binary_cross_entropy = 22.120689
Epoch 48
Validation binary_cross_entropy = 8.080283
Epoch 49
Loss = 2.1861e+01, PNorm = 53.2980, GNorm = 321.9467, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 3.160034
Epoch 50
Validation binary_cross_entropy = 2.524989
Epoch 51
Validation binary_cross_entropy = 9.188607
Epoch 52
Validation binary_cross_entropy = 38.666492
Epoch 53
Validation binary_cross_entropy = 4.339548
Epoch 54
Loss = 9.5496e+00, PNorm = 53.3426, GNorm = 566.7606, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 2.229835
Epoch 55
Validation binary_cross_entropy = 3.140361
Epoch 56
Validation binary_cross_entropy = 29.767380
Epoch 57
Validation binary_cross_entropy = 12.140307
Epoch 58
Validation binary_cross_entropy = 1.831889
Epoch 59
Loss = 8.1924e+00, PNorm = 53.3922, GNorm = 330.9132, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 2.650513
Epoch 60
Validation binary_cross_entropy = 4.383639
Epoch 61
Validation binary_cross_entropy = 30.071653
Epoch 62
Validation binary_cross_entropy = 15.636666
Epoch 63
Validation binary_cross_entropy = 1.603438
Epoch 64
Loss = 4.9376e+00, PNorm = 53.4379, GNorm = 164.7231, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 2.081970
Epoch 65
Validation binary_cross_entropy = 1.461870
Epoch 66
Validation binary_cross_entropy = 4.032168
Epoch 67
Validation binary_cross_entropy = 14.761887
Epoch 68
Validation binary_cross_entropy = 7.140884
Epoch 69
Loss = 5.2943e+00, PNorm = 53.4774, GNorm = 216.7838, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 1.924974
Epoch 70
Validation binary_cross_entropy = 1.602591
Epoch 71
Validation binary_cross_entropy = 1.873791
Epoch 72
Validation binary_cross_entropy = 1.908421
Epoch 73
Validation binary_cross_entropy = 2.720061
Epoch 74
Loss = 4.2387e+00, PNorm = 53.5123, GNorm = 194.2238, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 3.300298
Epoch 75
Validation binary_cross_entropy = 2.459891
Epoch 76
Validation binary_cross_entropy = 1.315069
Epoch 77
Validation binary_cross_entropy = 1.352453
Epoch 78
Validation binary_cross_entropy = 1.283636
Epoch 79
Loss = 4.7984e+00, PNorm = 53.5418, GNorm = 275.7898, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 2.783434
Epoch 80
Validation binary_cross_entropy = 5.442602
Epoch 81
Validation binary_cross_entropy = 2.978827
Epoch 82
Validation binary_cross_entropy = 0.988328
Epoch 83
Validation binary_cross_entropy = 0.768333
Epoch 84
Loss = 4.0415e+00, PNorm = 53.5703, GNorm = 145.1073, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.779713
Epoch 85
Validation binary_cross_entropy = 1.877276
Epoch 86
Validation binary_cross_entropy = 1.771098
Epoch 87
Validation binary_cross_entropy = 1.470678
Epoch 88
Validation binary_cross_entropy = 1.008460
Epoch 89
Loss = 2.7961e+00, PNorm = 53.5943, GNorm = 314.5857, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.738952
Epoch 90
Validation binary_cross_entropy = 0.630596
Epoch 91
Validation binary_cross_entropy = 0.685308
Epoch 92
Validation binary_cross_entropy = 0.707111
Epoch 93
Validation binary_cross_entropy = 0.784015
Epoch 94
Loss = 1.5837e+00, PNorm = 53.6145, GNorm = 111.1281, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 1.138908
Epoch 95
Validation binary_cross_entropy = 1.650588
Epoch 96
Validation binary_cross_entropy = 2.016154
Epoch 97
Validation binary_cross_entropy = 0.877505
Epoch 98
Validation binary_cross_entropy = 0.645690
Epoch 99
Loss = 2.8448e+00, PNorm = 53.6310, GNorm = 146.7774, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.630467
Epoch 100
Validation binary_cross_entropy = 0.768781
Epoch 101
Validation binary_cross_entropy = 0.884123
Epoch 102
Validation binary_cross_entropy = 1.266992
Epoch 103
Validation binary_cross_entropy = 1.159463
Epoch 104
Loss = 2.1717e+00, PNorm = 53.6468, GNorm = 190.9940, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.869279
Epoch 105
Validation binary_cross_entropy = 0.740683
Epoch 106
Validation binary_cross_entropy = 0.686469
Epoch 107
Validation binary_cross_entropy = 0.697984
Epoch 108
Validation binary_cross_entropy = 0.853623
Epoch 109
Loss = 1.5940e+00, PNorm = 53.6653, GNorm = 69.8173, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 1.340378
Epoch 110
Validation binary_cross_entropy = 1.728142
Epoch 111
Validation binary_cross_entropy = 0.779896
Epoch 112
Validation binary_cross_entropy = 0.490535
Epoch 113
Validation binary_cross_entropy = 0.507610
Epoch 114
Loss = 1.8040e+00, PNorm = 53.6834, GNorm = 124.9371, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.516921
Epoch 115
Validation binary_cross_entropy = 0.480333
Epoch 116
Validation binary_cross_entropy = 0.587649
Epoch 117
Validation binary_cross_entropy = 0.665638
Epoch 118
Validation binary_cross_entropy = 0.632914
Epoch 119
Loss = 1.0698e+00, PNorm = 53.6994, GNorm = 108.0805, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.592811
Epoch 120
Validation binary_cross_entropy = 0.535693
Epoch 121
Validation binary_cross_entropy = 0.524640
Epoch 122
Validation binary_cross_entropy = 0.529466
Epoch 123
Validation binary_cross_entropy = 0.554854
Epoch 124
Loss = 2.8836e+00, PNorm = 53.7142, GNorm = 167.1679, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.580938
Epoch 125
Validation binary_cross_entropy = 0.620457
Epoch 126
Validation binary_cross_entropy = 0.719988
Epoch 127
Validation binary_cross_entropy = 0.767413
Epoch 128
Validation binary_cross_entropy = 0.678644
Epoch 129
Loss = 2.0115e+00, PNorm = 53.7286, GNorm = 106.7059, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.591402
Epoch 130
Validation binary_cross_entropy = 0.549695
Epoch 131
Validation binary_cross_entropy = 0.526052
Epoch 132
Validation binary_cross_entropy = 0.581198
Epoch 133
Validation binary_cross_entropy = 0.733602
Epoch 134
Loss = 2.7920e+00, PNorm = 53.7437, GNorm = 119.8838, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.458028
Epoch 135
Validation binary_cross_entropy = 0.403101
Epoch 136
Validation binary_cross_entropy = 0.429677
Epoch 137
Validation binary_cross_entropy = 0.350936
Epoch 138
Validation binary_cross_entropy = 0.418077
Epoch 139
Loss = 1.6037e+00, PNorm = 53.7581, GNorm = 157.8650, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.527518
Epoch 140
Validation binary_cross_entropy = 0.445425
Epoch 141
Validation binary_cross_entropy = 0.387833
Epoch 142
Validation binary_cross_entropy = 0.382040
Epoch 143
Validation binary_cross_entropy = 0.397796
Epoch 144
Loss = 1.2700e+00, PNorm = 53.7708, GNorm = 79.7829, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.418677
Epoch 145
Validation binary_cross_entropy = 0.424751
Epoch 146
Validation binary_cross_entropy = 0.564689
Epoch 147
Validation binary_cross_entropy = 0.716986
Epoch 148
Validation binary_cross_entropy = 0.714629
Epoch 149
Loss = 1.3107e+00, PNorm = 53.7821, GNorm = 72.8715, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.647701
Epoch 150
Validation binary_cross_entropy = 0.549625
Epoch 151
Validation binary_cross_entropy = 0.461343
Epoch 152
Validation binary_cross_entropy = 0.430312
Epoch 153
Validation binary_cross_entropy = 0.418521
Epoch 154
Loss = 1.2353e+00, PNorm = 53.7920, GNorm = 81.9846, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.413597
Epoch 155
Validation binary_cross_entropy = 0.444422
Epoch 156
Validation binary_cross_entropy = 0.563792
Epoch 157
Validation binary_cross_entropy = 0.687235
Epoch 158
Validation binary_cross_entropy = 0.587821
Epoch 159
Loss = 9.3084e-01, PNorm = 53.8011, GNorm = 84.9183, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.532522
Epoch 160
Validation binary_cross_entropy = 0.498490
Epoch 161
Validation binary_cross_entropy = 0.464162
Epoch 162
Validation binary_cross_entropy = 0.442617
Epoch 163
Validation binary_cross_entropy = 0.448858
Epoch 164
Loss = 9.1616e-01, PNorm = 53.8094, GNorm = 62.7833, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.462892
Epoch 165
Validation binary_cross_entropy = 0.476080
Epoch 166
Validation binary_cross_entropy = 0.511146
Epoch 167
Validation binary_cross_entropy = 0.527122
Epoch 168
Validation binary_cross_entropy = 0.513614
Epoch 169
Loss = 1.1285e+00, PNorm = 53.8174, GNorm = 61.2301, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.484524
Epoch 170
Validation binary_cross_entropy = 0.450109
Epoch 171
Validation binary_cross_entropy = 0.435088
Epoch 172
Validation binary_cross_entropy = 0.441594
Epoch 173
Validation binary_cross_entropy = 0.471396
Epoch 174
Loss = 5.4792e-01, PNorm = 53.8261, GNorm = 60.7399, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.519933
Epoch 175
Validation binary_cross_entropy = 0.637040
Epoch 176
Validation binary_cross_entropy = 0.609016
Epoch 177
Validation binary_cross_entropy = 0.554260
Epoch 178
Validation binary_cross_entropy = 0.466695
Epoch 179
Loss = 1.0493e+00, PNorm = 53.8347, GNorm = 34.9170, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.452761
Epoch 180
Validation binary_cross_entropy = 0.476160
Epoch 181
Validation binary_cross_entropy = 0.511013
Epoch 182
Validation binary_cross_entropy = 0.549177
Epoch 183
Validation binary_cross_entropy = 0.582259
Epoch 184
Loss = 8.1439e-01, PNorm = 53.8430, GNorm = 55.9451, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.598809
Epoch 185
Validation binary_cross_entropy = 0.603392
Epoch 186
Validation binary_cross_entropy = 0.567114
Epoch 187
Validation binary_cross_entropy = 0.545143
Epoch 188
Validation binary_cross_entropy = 0.551034
Epoch 189
Loss = 1.1142e+00, PNorm = 53.8499, GNorm = 83.1550, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.565309
Epoch 190
Validation binary_cross_entropy = 0.594669
Epoch 191
Validation binary_cross_entropy = 0.571583
Epoch 192
Validation binary_cross_entropy = 0.545677
Epoch 193
Validation binary_cross_entropy = 0.526315
Epoch 194
Loss = 6.9796e-01, PNorm = 53.8559, GNorm = 31.4304, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.526236
Epoch 195
Validation binary_cross_entropy = 0.508213
Epoch 196
Validation binary_cross_entropy = 0.487893
Epoch 197
Validation binary_cross_entropy = 0.462848
Epoch 198
Validation binary_cross_entropy = 0.446515
Epoch 199
Loss = 9.8015e-01, PNorm = 53.8611, GNorm = 90.7294, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.435973
Epoch 200
Validation binary_cross_entropy = 0.427136
Epoch 201
Validation binary_cross_entropy = 0.390551
Epoch 202
Validation binary_cross_entropy = 0.396047
Epoch 203
Validation binary_cross_entropy = 0.515090
Epoch 204
Loss = 6.5715e-01, PNorm = 53.8667, GNorm = 31.1806, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.546814
Epoch 205
Validation binary_cross_entropy = 0.520031
Epoch 206
Validation binary_cross_entropy = 0.532986
Epoch 207
Validation binary_cross_entropy = 0.506242
Epoch 208
Validation binary_cross_entropy = 0.466860
Epoch 209
Loss = 5.0230e-01, PNorm = 53.8732, GNorm = 23.0699, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.376252
Epoch 210
Validation binary_cross_entropy = 0.345354
Epoch 211
Validation binary_cross_entropy = 0.357844
Epoch 212
Validation binary_cross_entropy = 0.418466
Epoch 213
Validation binary_cross_entropy = 0.463309
Epoch 214
Loss = 8.3738e-01, PNorm = 53.8791, GNorm = 40.9535, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.492534
Epoch 215
Validation binary_cross_entropy = 0.477785
Epoch 216
Validation binary_cross_entropy = 0.458131
Epoch 217
Validation binary_cross_entropy = 0.429451
Epoch 218
Validation binary_cross_entropy = 0.409992
Epoch 219
Loss = 1.1507e+00, PNorm = 53.8845, GNorm = 57.8622, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.400827
Epoch 220
Validation binary_cross_entropy = 0.419255
Epoch 221
Validation binary_cross_entropy = 0.448788
Epoch 222
Validation binary_cross_entropy = 0.487562
Epoch 223
Validation binary_cross_entropy = 0.531845
Epoch 224
Loss = 1.0131e+00, PNorm = 53.8910, GNorm = 66.0473, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.609309
Epoch 225
Validation binary_cross_entropy = 0.620606
Epoch 226
Validation binary_cross_entropy = 0.579320
Epoch 227
Validation binary_cross_entropy = 0.517184
Epoch 228
Validation binary_cross_entropy = 0.464017
Epoch 229
Loss = 6.1147e-01, PNorm = 53.8969, GNorm = 50.8391, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.435429
Epoch 230
Validation binary_cross_entropy = 0.437019
Epoch 231
Validation binary_cross_entropy = 0.441631
Epoch 232
Validation binary_cross_entropy = 0.452842
Epoch 233
Validation binary_cross_entropy = 0.477276
Epoch 234
Loss = 4.2627e-01, PNorm = 53.9019, GNorm = 40.7500, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.513368
Epoch 235
Validation binary_cross_entropy = 0.552209
Epoch 236
Validation binary_cross_entropy = 0.564420
Epoch 237
Validation binary_cross_entropy = 0.550663
Epoch 238
Validation binary_cross_entropy = 0.513492
Epoch 239
Loss = 6.4180e-01, PNorm = 53.9065, GNorm = 49.1631, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.472882
Epoch 240
Validation binary_cross_entropy = 0.440658
Epoch 241
Validation binary_cross_entropy = 0.430037
Epoch 242
Validation binary_cross_entropy = 0.436732
Epoch 243
Validation binary_cross_entropy = 0.471194
Epoch 244
Loss = 7.2125e-01, PNorm = 53.9111, GNorm = 99.2676, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.527629
Epoch 245
Validation binary_cross_entropy = 0.521487
Epoch 246
Validation binary_cross_entropy = 0.486056
Epoch 247
Validation binary_cross_entropy = 0.449584
Epoch 248
Validation binary_cross_entropy = 0.421686
Epoch 249
Loss = 6.2102e-01, PNorm = 53.9158, GNorm = 53.5759, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.416110
Epoch 250
Validation binary_cross_entropy = 0.455613
Epoch 251
Validation binary_cross_entropy = 0.570356
Epoch 252
Validation binary_cross_entropy = 0.504518
Epoch 253
Validation binary_cross_entropy = 0.452376
Epoch 254
Loss = 7.2430e-01, PNorm = 53.9201, GNorm = 41.4449, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.416793
Epoch 255
Validation binary_cross_entropy = 0.409938
Epoch 256
Validation binary_cross_entropy = 0.410320
Epoch 257
Validation binary_cross_entropy = 0.428077
Epoch 258
Validation binary_cross_entropy = 0.448946
Epoch 259
Loss = 6.5498e-01, PNorm = 53.9247, GNorm = 62.3619, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.448612
Epoch 260
Validation binary_cross_entropy = 0.448372
Epoch 261
Validation binary_cross_entropy = 0.443636
Epoch 262
Validation binary_cross_entropy = 0.439663
Epoch 263
Validation binary_cross_entropy = 0.441218
Epoch 264
Loss = 6.8472e-01, PNorm = 53.9290, GNorm = 65.6569, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.450499
Epoch 265
Validation binary_cross_entropy = 0.460964
Epoch 266
Validation binary_cross_entropy = 0.459774
Epoch 267
Validation binary_cross_entropy = 0.477934
Epoch 268
Validation binary_cross_entropy = 0.464601
Epoch 269
Loss = 5.6070e-01, PNorm = 53.9335, GNorm = 27.8257, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.471356
Epoch 270
Validation binary_cross_entropy = 0.507356
Epoch 271
Validation binary_cross_entropy = 0.523595
Epoch 272
Validation binary_cross_entropy = 0.481072
Epoch 273
Validation binary_cross_entropy = 0.463077
Epoch 274
Loss = 1.0287e+00, PNorm = 53.9381, GNorm = 65.1000, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.444084
Epoch 275
Validation binary_cross_entropy = 0.432009
Epoch 276
Validation binary_cross_entropy = 0.463910
Epoch 277
Validation binary_cross_entropy = 0.531439
Epoch 278
Validation binary_cross_entropy = 0.488449
Epoch 279
Loss = 5.5377e-01, PNorm = 53.9426, GNorm = 25.1614, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.444595
Epoch 280
Validation binary_cross_entropy = 0.416422
Epoch 281
Validation binary_cross_entropy = 0.412331
Epoch 282
Validation binary_cross_entropy = 0.438747
Epoch 283
Validation binary_cross_entropy = 0.459081
Epoch 284
Loss = 5.9336e-01, PNorm = 53.9474, GNorm = 49.4768, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.451997
Epoch 285
Validation binary_cross_entropy = 0.463644
Epoch 286
Validation binary_cross_entropy = 0.431836
Epoch 287
Validation binary_cross_entropy = 0.424134
Epoch 288
Validation binary_cross_entropy = 0.435853
Epoch 289
Loss = 6.9573e-01, PNorm = 53.9524, GNorm = 42.5186, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.454204
Epoch 290
Validation binary_cross_entropy = 0.457604
Epoch 291
Validation binary_cross_entropy = 0.460604
Epoch 292
Validation binary_cross_entropy = 0.445865
Epoch 293
Validation binary_cross_entropy = 0.435555
Epoch 294
Loss = 7.0433e-01, PNorm = 53.9577, GNorm = 57.2618, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.435109
Epoch 295
Validation binary_cross_entropy = 0.420416
Epoch 296
Validation binary_cross_entropy = 0.413460
Epoch 297
Validation binary_cross_entropy = 0.393193
Epoch 298
Validation binary_cross_entropy = 0.396239
Epoch 299
Loss = 5.9017e-01, PNorm = 53.9634, GNorm = 80.5508, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.425379
Model 0 best validation binary_cross_entropy = 0.345354 on epoch 210
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.369915
Ensemble test binary_cross_entropy = 0.369915
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=300, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.35000000000000003, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 5,506,601
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 90.445862
Epoch 1
Validation binary_cross_entropy = 7.961690
Epoch 2
Validation binary_cross_entropy = 50.814579
Epoch 3
Validation binary_cross_entropy = 10.781758
Epoch 4
Loss = 9.3619e+01, PNorm = 52.2775, GNorm = 1526.4716, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 7.851397
Epoch 5
Validation binary_cross_entropy = 41.881294
Epoch 6
Validation binary_cross_entropy = 10.167603
Epoch 7
Validation binary_cross_entropy = 9.566705
Epoch 8
Validation binary_cross_entropy = 7.174634
Epoch 9
Loss = 4.7275e+01, PNorm = 52.4681, GNorm = 911.0619, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 10.607551
Epoch 10
Validation binary_cross_entropy = 7.688651
Epoch 11
Validation binary_cross_entropy = 18.266136
Epoch 12
Validation binary_cross_entropy = 9.283516
Epoch 13
Validation binary_cross_entropy = 7.911890
Epoch 14
Loss = 5.1561e+01, PNorm = 52.7084, GNorm = 738.9956, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 19.155809
Epoch 15
Validation binary_cross_entropy = 6.589033
Epoch 16
Validation binary_cross_entropy = 19.796568
Epoch 17
Validation binary_cross_entropy = 15.247522
Epoch 18
Validation binary_cross_entropy = 8.365156
Epoch 19
Loss = 4.5821e+01, PNorm = 52.9058, GNorm = 1376.2468, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 9.406484
Epoch 20
Validation binary_cross_entropy = 6.733358
Epoch 21
Validation binary_cross_entropy = 73.487305
Epoch 22
Validation binary_cross_entropy = 8.278872
Epoch 23
Validation binary_cross_entropy = 16.135616
Epoch 24
Loss = 3.5434e+01, PNorm = 53.0680, GNorm = 1455.1063, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 45.874973
Epoch 25
Validation binary_cross_entropy = 22.696812
Epoch 26
Validation binary_cross_entropy = 13.856746
Epoch 27
Validation binary_cross_entropy = 15.038217
Epoch 28
Validation binary_cross_entropy = 13.663581
Epoch 29
Loss = 2.8529e+01, PNorm = 53.2012, GNorm = 745.5764, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 14.821047
Epoch 30
Validation binary_cross_entropy = 21.930302
Epoch 31
Validation binary_cross_entropy = 15.778087
Epoch 32
Validation binary_cross_entropy = 17.040300
Epoch 33
Validation binary_cross_entropy = 12.747540
Epoch 34
Loss = 2.2421e+01, PNorm = 53.3035, GNorm = 801.7810, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 27.989248
Epoch 35
Validation binary_cross_entropy = 35.262260
Epoch 36
Validation binary_cross_entropy = 8.445375
Epoch 37
Validation binary_cross_entropy = 13.620828
Epoch 38
Validation binary_cross_entropy = 28.573296
Epoch 39
Loss = 2.5603e+01, PNorm = 53.3834, GNorm = 587.7922, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 11.932126
Epoch 40
Validation binary_cross_entropy = 8.019293
Epoch 41
Validation binary_cross_entropy = 11.166419
Epoch 42
Validation binary_cross_entropy = 11.685221
Epoch 43
Validation binary_cross_entropy = 6.104526
Epoch 44
Loss = 2.4359e+01, PNorm = 53.4500, GNorm = 1270.3052, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 13.789486
Epoch 45
Validation binary_cross_entropy = 72.012581
Epoch 46
Validation binary_cross_entropy = 19.868279
Epoch 47
Validation binary_cross_entropy = 4.660391
Epoch 48
Validation binary_cross_entropy = 4.071886
Epoch 49
Loss = 2.4407e+01, PNorm = 53.5149, GNorm = 296.9287, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 12.399439
Epoch 50
Validation binary_cross_entropy = 22.395849
Epoch 51
Validation binary_cross_entropy = 9.377378
Epoch 52
Validation binary_cross_entropy = 4.943868
Epoch 53
Validation binary_cross_entropy = 5.083492
Epoch 54
Loss = 1.6502e+01, PNorm = 53.5677, GNorm = 376.8447, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 14.642628
Epoch 55
Validation binary_cross_entropy = 38.550583
Epoch 56
Validation binary_cross_entropy = 21.804115
Epoch 57
Validation binary_cross_entropy = 7.163571
Epoch 58
Validation binary_cross_entropy = 13.622571
Epoch 59
Loss = 6.9908e+00, PNorm = 53.6106, GNorm = 251.6189, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 26.675846
Epoch 60
Validation binary_cross_entropy = 12.346090
Epoch 61
Validation binary_cross_entropy = 5.339396
Epoch 62
Validation binary_cross_entropy = 3.959086
Epoch 63
Validation binary_cross_entropy = 3.814765
Epoch 64
Loss = 1.0194e+01, PNorm = 53.6486, GNorm = 184.3295, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 10.291139
Epoch 65
Validation binary_cross_entropy = 11.885529
Epoch 66
Validation binary_cross_entropy = 4.075090
Epoch 67
Validation binary_cross_entropy = 3.004381
Epoch 68
Validation binary_cross_entropy = 3.797042
Epoch 69
Loss = 8.1135e+00, PNorm = 53.6861, GNorm = 150.0449, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 15.807390
Epoch 70
Validation binary_cross_entropy = 22.594854
Epoch 71
Validation binary_cross_entropy = 5.128219
Epoch 72
Validation binary_cross_entropy = 3.108844
Epoch 73
Validation binary_cross_entropy = 3.248059
Epoch 74
Loss = 8.7541e+00, PNorm = 53.7202, GNorm = 484.6899, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 2.812439
Epoch 75
Validation binary_cross_entropy = 9.145871
Epoch 76
Validation binary_cross_entropy = 20.494570
Epoch 77
Validation binary_cross_entropy = 8.256724
Epoch 78
Validation binary_cross_entropy = 2.661922
Epoch 79
Loss = 6.8460e+00, PNorm = 53.7503, GNorm = 413.2736, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 2.547017
Epoch 80
Validation binary_cross_entropy = 2.417875
Epoch 81
Validation binary_cross_entropy = 4.512095
Epoch 82
Validation binary_cross_entropy = 8.613503
Epoch 83
Validation binary_cross_entropy = 7.595803
Epoch 84
Loss = 4.4823e+00, PNorm = 53.7744, GNorm = 219.5451, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 3.502319
Epoch 85
Validation binary_cross_entropy = 1.380541
Epoch 86
Validation binary_cross_entropy = 1.660220
Epoch 87
Validation binary_cross_entropy = 1.389911
Epoch 88
Validation binary_cross_entropy = 1.630178
Epoch 89
Loss = 4.9545e+00, PNorm = 53.7946, GNorm = 154.8255, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 3.055479
Epoch 90
Validation binary_cross_entropy = 2.926259
Epoch 91
Validation binary_cross_entropy = 1.625379
Epoch 92
Validation binary_cross_entropy = 1.239708
Epoch 93
Validation binary_cross_entropy = 1.176965
Epoch 94
Loss = 4.3798e+00, PNorm = 53.8148, GNorm = 164.9369, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 1.106822
Epoch 95
Validation binary_cross_entropy = 1.960639
Epoch 96
Validation binary_cross_entropy = 3.423311
Epoch 97
Validation binary_cross_entropy = 4.512572
Epoch 98
Validation binary_cross_entropy = 3.787393
Epoch 99
Loss = 4.7784e+00, PNorm = 53.8350, GNorm = 173.1714, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 1.173111
Epoch 100
Validation binary_cross_entropy = 0.979229
Epoch 101
Validation binary_cross_entropy = 1.061615
Epoch 102
Validation binary_cross_entropy = 0.921111
Epoch 103
Validation binary_cross_entropy = 1.556911
Epoch 104
Loss = 2.6481e+00, PNorm = 53.8557, GNorm = 202.1766, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 3.019549
Epoch 105
Validation binary_cross_entropy = 1.645527
Epoch 106
Validation binary_cross_entropy = 1.105726
Epoch 107
Validation binary_cross_entropy = 1.065630
Epoch 108
Validation binary_cross_entropy = 1.723535
Epoch 109
Loss = 2.6823e+00, PNorm = 53.8761, GNorm = 104.2048, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 1.527478
Epoch 110
Validation binary_cross_entropy = 1.561847
Epoch 111
Validation binary_cross_entropy = 0.797003
Epoch 112
Validation binary_cross_entropy = 0.919955
Epoch 113
Validation binary_cross_entropy = 1.492271
Epoch 114
Loss = 3.4820e+00, PNorm = 53.8951, GNorm = 150.7271, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 1.027705
Epoch 115
Validation binary_cross_entropy = 0.711912
Epoch 116
Validation binary_cross_entropy = 0.747433
Epoch 117
Validation binary_cross_entropy = 1.137135
Epoch 118
Validation binary_cross_entropy = 2.445533
Epoch 119
Loss = 2.5304e+00, PNorm = 53.9122, GNorm = 183.5984, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 3.209744
Epoch 120
Validation binary_cross_entropy = 1.383412
Epoch 121
Validation binary_cross_entropy = 0.882069
Epoch 122
Validation binary_cross_entropy = 0.885286
Epoch 123
Validation binary_cross_entropy = 1.010927
Epoch 124
Loss = 2.2747e+00, PNorm = 53.9288, GNorm = 143.4442, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 1.311455
Epoch 125
Validation binary_cross_entropy = 2.848020
Epoch 126
Validation binary_cross_entropy = 2.562413
Epoch 127
Validation binary_cross_entropy = 1.120690
Epoch 128
Validation binary_cross_entropy = 0.598036
Epoch 129
Loss = 1.4945e+00, PNorm = 53.9454, GNorm = 159.8516, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.498018
Epoch 130
Validation binary_cross_entropy = 0.524044
Epoch 131
Validation binary_cross_entropy = 0.650645
Epoch 132
Validation binary_cross_entropy = 0.537795
Epoch 133
Validation binary_cross_entropy = 0.577562
Epoch 134
Loss = 1.2007e+00, PNorm = 53.9602, GNorm = 101.5960, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.750582
Epoch 135
Validation binary_cross_entropy = 0.974554
Epoch 136
Validation binary_cross_entropy = 0.893134
Epoch 137
Validation binary_cross_entropy = 0.814911
Epoch 138
Validation binary_cross_entropy = 0.683044
Epoch 139
Loss = 8.7978e-01, PNorm = 53.9745, GNorm = 99.1955, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.693749
Epoch 140
Validation binary_cross_entropy = 0.939052
Epoch 141
Validation binary_cross_entropy = 1.467290
Epoch 142
Validation binary_cross_entropy = 1.342621
Epoch 143
Validation binary_cross_entropy = 0.641125
Epoch 144
Loss = 1.5306e+00, PNorm = 53.9870, GNorm = 102.3111, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.440915
Epoch 145
Validation binary_cross_entropy = 0.423956
Epoch 146
Validation binary_cross_entropy = 0.436613
Epoch 147
Validation binary_cross_entropy = 0.462085
Epoch 148
Validation binary_cross_entropy = 0.535376
Epoch 149
Loss = 1.4383e+00, PNorm = 53.9978, GNorm = 155.0706, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.637947
Epoch 150
Validation binary_cross_entropy = 0.569532
Epoch 151
Validation binary_cross_entropy = 0.515244
Epoch 152
Validation binary_cross_entropy = 0.547781
Epoch 153
Validation binary_cross_entropy = 0.618617
Epoch 154
Loss = 1.4098e+00, PNorm = 54.0095, GNorm = 107.3221, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.646856
Epoch 155
Validation binary_cross_entropy = 0.667386
Epoch 156
Validation binary_cross_entropy = 0.608375
Epoch 157
Validation binary_cross_entropy = 0.537945
Epoch 158
Validation binary_cross_entropy = 0.562418
Epoch 159
Loss = 1.1006e+00, PNorm = 54.0203, GNorm = 64.0232, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.693375
Epoch 160
Validation binary_cross_entropy = 0.851155
Epoch 161
Validation binary_cross_entropy = 0.689149
Epoch 162
Validation binary_cross_entropy = 0.576410
Epoch 163
Validation binary_cross_entropy = 0.540998
Epoch 164
Loss = 2.2956e+00, PNorm = 54.0296, GNorm = 125.0055, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.551292
Epoch 165
Validation binary_cross_entropy = 0.595599
Epoch 166
Validation binary_cross_entropy = 0.612016
Epoch 167
Validation binary_cross_entropy = 0.613458
Epoch 168
Validation binary_cross_entropy = 0.678724
Epoch 169
Loss = 1.1595e+00, PNorm = 54.0386, GNorm = 92.3112, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.785663
Epoch 170
Validation binary_cross_entropy = 0.771001
Epoch 171
Validation binary_cross_entropy = 0.644714
Epoch 172
Validation binary_cross_entropy = 0.579526
Epoch 173
Validation binary_cross_entropy = 0.562201
Epoch 174
Loss = 1.5514e+00, PNorm = 54.0472, GNorm = 80.8918, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.564276
Epoch 175
Validation binary_cross_entropy = 0.587299
Epoch 176
Validation binary_cross_entropy = 0.602732
Epoch 177
Validation binary_cross_entropy = 0.588634
Epoch 178
Validation binary_cross_entropy = 0.567800
Epoch 179
Loss = 7.6802e-01, PNorm = 54.0557, GNorm = 73.4774, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.536562
Epoch 180
Validation binary_cross_entropy = 0.489797
Epoch 181
Validation binary_cross_entropy = 0.452783
Epoch 182
Validation binary_cross_entropy = 0.449184
Epoch 183
Validation binary_cross_entropy = 0.469917
Epoch 184
Loss = 6.8129e-01, PNorm = 54.0635, GNorm = 66.8629, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.560097
Epoch 185
Validation binary_cross_entropy = 0.644222
Epoch 186
Validation binary_cross_entropy = 0.618727
Epoch 187
Validation binary_cross_entropy = 0.570060
Epoch 188
Validation binary_cross_entropy = 0.696052
Epoch 189
Loss = 8.0597e-01, PNorm = 54.0713, GNorm = 77.1596, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.563046
Epoch 190
Validation binary_cross_entropy = 0.420149
Epoch 191
Validation binary_cross_entropy = 0.434272
Epoch 192
Validation binary_cross_entropy = 0.484960
Epoch 193
Validation binary_cross_entropy = 0.451595
Epoch 194
Loss = 6.2999e-01, PNorm = 54.0791, GNorm = 67.8414, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.458673
Epoch 195
Validation binary_cross_entropy = 0.427741
Epoch 196
Validation binary_cross_entropy = 0.407708
Epoch 197
Validation binary_cross_entropy = 0.377954
Epoch 198
Validation binary_cross_entropy = 0.424511
Epoch 199
Loss = 9.0338e-01, PNorm = 54.0865, GNorm = 186.0804, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.446006
Epoch 200
Validation binary_cross_entropy = 0.420494
Epoch 201
Validation binary_cross_entropy = 0.421606
Epoch 202
Validation binary_cross_entropy = 0.411913
Epoch 203
Validation binary_cross_entropy = 0.436243
Epoch 204
Loss = 5.4941e-01, PNorm = 54.0934, GNorm = 29.9255, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.432193
Epoch 205
Validation binary_cross_entropy = 0.463130
Epoch 206
Validation binary_cross_entropy = 0.509468
Epoch 207
Validation binary_cross_entropy = 0.563945
Epoch 208
Validation binary_cross_entropy = 0.626715
Epoch 209
Loss = 7.5567e-01, PNorm = 54.0999, GNorm = 67.1406, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.600603
Epoch 210
Validation binary_cross_entropy = 0.539720
Epoch 211
Validation binary_cross_entropy = 0.477042
Epoch 212
Validation binary_cross_entropy = 0.447354
Epoch 213
Validation binary_cross_entropy = 0.434969
Epoch 214
Loss = 5.6438e-01, PNorm = 54.1051, GNorm = 58.3377, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.419816
Epoch 215
Validation binary_cross_entropy = 0.436611
Epoch 216
Validation binary_cross_entropy = 0.481134
Epoch 217
Validation binary_cross_entropy = 0.537129
Epoch 218
Validation binary_cross_entropy = 0.535885
Epoch 219
Loss = 8.1926e-01, PNorm = 54.1101, GNorm = 45.2900, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.509242
Epoch 220
Validation binary_cross_entropy = 0.485360
Epoch 221
Validation binary_cross_entropy = 0.456971
Epoch 222
Validation binary_cross_entropy = 0.460901
Epoch 223
Validation binary_cross_entropy = 0.456645
Epoch 224
Loss = 7.5787e-01, PNorm = 54.1142, GNorm = 55.0251, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.454151
Epoch 225
Validation binary_cross_entropy = 0.457535
Epoch 226
Validation binary_cross_entropy = 0.488163
Epoch 227
Validation binary_cross_entropy = 0.522321
Epoch 228
Validation binary_cross_entropy = 0.537115
Epoch 229
Loss = 6.8206e-01, PNorm = 54.1186, GNorm = 57.9330, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.542154
Epoch 230
Validation binary_cross_entropy = 0.525800
Epoch 231
Validation binary_cross_entropy = 0.514758
Epoch 232
Validation binary_cross_entropy = 0.537468
Epoch 233
Validation binary_cross_entropy = 0.554391
Epoch 234
Loss = 5.5781e-01, PNorm = 54.1228, GNorm = 47.5209, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.513471
Epoch 235
Validation binary_cross_entropy = 0.494477
Epoch 236
Validation binary_cross_entropy = 0.504349
Epoch 237
Validation binary_cross_entropy = 0.528816
Epoch 238
Validation binary_cross_entropy = 0.552462
Epoch 239
Loss = 5.6900e-01, PNorm = 54.1278, GNorm = 156.9019, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.579297
Epoch 240
Validation binary_cross_entropy = 0.589593
Epoch 241
Validation binary_cross_entropy = 0.585276
Epoch 242
Validation binary_cross_entropy = 0.565508
Epoch 243
Validation binary_cross_entropy = 0.525623
Epoch 244
Loss = 6.2009e-01, PNorm = 54.1331, GNorm = 101.1759, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.481301
Epoch 245
Validation binary_cross_entropy = 0.470531
Epoch 246
Validation binary_cross_entropy = 0.426988
Epoch 247
Validation binary_cross_entropy = 0.419630
Epoch 248
Validation binary_cross_entropy = 0.439661
Epoch 249
Loss = 5.8170e-01, PNorm = 54.1377, GNorm = 45.6529, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.470345
Epoch 250
Validation binary_cross_entropy = 0.508101
Epoch 251
Validation binary_cross_entropy = 0.546032
Epoch 252
Validation binary_cross_entropy = 0.569086
Epoch 253
Validation binary_cross_entropy = 0.527264
Epoch 254
Loss = 5.9920e-01, PNorm = 54.1414, GNorm = 49.2332, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.465204
Epoch 255
Validation binary_cross_entropy = 0.424802
Epoch 256
Validation binary_cross_entropy = 0.425357
Epoch 257
Validation binary_cross_entropy = 0.427453
Epoch 258
Validation binary_cross_entropy = 0.411998
Epoch 259
Loss = 5.2312e-01, PNorm = 54.1455, GNorm = 49.8308, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.417964
Epoch 260
Validation binary_cross_entropy = 0.476126
Epoch 261
Validation binary_cross_entropy = 0.530085
Epoch 262
Validation binary_cross_entropy = 0.563516
Epoch 263
Validation binary_cross_entropy = 0.611927
Epoch 264
Loss = 7.7129e-01, PNorm = 54.1499, GNorm = 48.0070, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.677121
Epoch 265
Validation binary_cross_entropy = 0.633358
Epoch 266
Validation binary_cross_entropy = 0.491160
Epoch 267
Validation binary_cross_entropy = 0.405728
Epoch 268
Validation binary_cross_entropy = 0.383561
Epoch 269
Loss = 9.1669e-01, PNorm = 54.1548, GNorm = 52.1478, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.412532
Epoch 270
Validation binary_cross_entropy = 0.472097
Epoch 271
Validation binary_cross_entropy = 0.513675
Epoch 272
Validation binary_cross_entropy = 0.529428
Epoch 273
Validation binary_cross_entropy = 0.524856
Epoch 274
Loss = 5.2064e-01, PNorm = 54.1594, GNorm = 63.1977, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.465222
Epoch 275
Validation binary_cross_entropy = 0.421687
Epoch 276
Validation binary_cross_entropy = 0.407724
Epoch 277
Validation binary_cross_entropy = 0.398609
Epoch 278
Validation binary_cross_entropy = 0.404235
Epoch 279
Loss = 5.3489e-01, PNorm = 54.1638, GNorm = 43.2136, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.482551
Epoch 280
Validation binary_cross_entropy = 0.532340
Epoch 281
Validation binary_cross_entropy = 0.513630
Epoch 282
Validation binary_cross_entropy = 0.515084
Epoch 283
Validation binary_cross_entropy = 0.502651
Epoch 284
Loss = 6.0950e-01, PNorm = 54.1673, GNorm = 54.0885, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.474269
Epoch 285
Validation binary_cross_entropy = 0.492161
Epoch 286
Validation binary_cross_entropy = 0.543074
Epoch 287
Validation binary_cross_entropy = 0.529844
Epoch 288
Validation binary_cross_entropy = 0.505118
Epoch 289
Loss = 3.8256e-01, PNorm = 54.1703, GNorm = 26.0961, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.504334
Epoch 290
Validation binary_cross_entropy = 0.486669
Epoch 291
Validation binary_cross_entropy = 0.462280
Epoch 292
Validation binary_cross_entropy = 0.453222
Epoch 293
Validation binary_cross_entropy = 0.477718
Epoch 294
Loss = 5.9067e-01, PNorm = 54.1733, GNorm = 61.3003, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.507805
Epoch 295
Validation binary_cross_entropy = 0.542296
Epoch 296
Validation binary_cross_entropy = 0.550682
Epoch 297
Validation binary_cross_entropy = 0.553198
Epoch 298
Validation binary_cross_entropy = 0.540560
Epoch 299
Loss = 4.9957e-01, PNorm = 54.1763, GNorm = 32.3706, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.541149
Model 0 best validation binary_cross_entropy = 0.377954 on epoch 197
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.422743
Ensemble test binary_cross_entropy = 0.422743
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=300, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.35000000000000003, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 5,506,601
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 11.318839
Epoch 1
Validation binary_cross_entropy = 54.675690
Epoch 2
Validation binary_cross_entropy = 8.551402
Epoch 3
Validation binary_cross_entropy = 12.293302
Epoch 4
Loss = 9.8933e+01, PNorm = 52.2836, GNorm = 2359.8172, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 85.653267
Epoch 5
Validation binary_cross_entropy = 6.378992
Epoch 6
Validation binary_cross_entropy = 33.313408
Epoch 7
Validation binary_cross_entropy = 18.964083
Epoch 8
Validation binary_cross_entropy = 9.431330
Epoch 9
Loss = 8.8981e+01, PNorm = 52.4904, GNorm = 743.2536, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 68.292473
Epoch 10
Validation binary_cross_entropy = 52.393871
Epoch 11
Validation binary_cross_entropy = 9.693317
Epoch 12
Validation binary_cross_entropy = 9.760429
Epoch 13
Validation binary_cross_entropy = 37.745987
Epoch 14
Loss = 1.2609e+02, PNorm = 52.7375, GNorm = 2478.3341, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 30.268244
Epoch 15
Validation binary_cross_entropy = 8.232815
Epoch 16
Validation binary_cross_entropy = 7.798717
Epoch 17
Validation binary_cross_entropy = 12.290004
Epoch 18
Validation binary_cross_entropy = 40.623901
Epoch 19
Loss = 5.1394e+01, PNorm = 52.9289, GNorm = 928.2182, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 37.090603
Epoch 20
Validation binary_cross_entropy = 6.818727
Epoch 21
Validation binary_cross_entropy = 5.340198
Epoch 22
Validation binary_cross_entropy = 4.981339
Epoch 23
Validation binary_cross_entropy = 8.468019
Epoch 24
Loss = 1.5448e+01, PNorm = 53.0608, GNorm = 433.1970, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 17.830593
Epoch 25
Validation binary_cross_entropy = 8.418426
Epoch 26
Validation binary_cross_entropy = 4.517389
Epoch 27
Validation binary_cross_entropy = 4.155434
Epoch 28
Validation binary_cross_entropy = 4.264609
Epoch 29
Loss = 2.0911e+01, PNorm = 53.1467, GNorm = 418.1672, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 7.656816
Epoch 30
Validation binary_cross_entropy = 8.673848
Epoch 31
Validation binary_cross_entropy = 6.610891
Epoch 32
Validation binary_cross_entropy = 5.192982
Epoch 33
Validation binary_cross_entropy = 4.702992
Epoch 34
Loss = 1.0117e+01, PNorm = 53.2090, GNorm = 251.6271, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 9.997346
Epoch 35
Validation binary_cross_entropy = 11.737743
Epoch 36
Validation binary_cross_entropy = 4.812377
Epoch 37
Validation binary_cross_entropy = 3.167094
Epoch 38
Validation binary_cross_entropy = 5.072544
Epoch 39
Loss = 1.4492e+01, PNorm = 53.2608, GNorm = 440.1842, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 7.949813
Epoch 40
Validation binary_cross_entropy = 3.956972
Epoch 41
Validation binary_cross_entropy = 3.329208
Epoch 42
Validation binary_cross_entropy = 5.165983
Epoch 43
Validation binary_cross_entropy = 7.086164
Epoch 44
Loss = 1.4823e+01, PNorm = 53.3086, GNorm = 421.8151, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 10.354776
Epoch 45
Validation binary_cross_entropy = 8.351547
Epoch 46
Validation binary_cross_entropy = 10.523532
Epoch 47
Validation binary_cross_entropy = 10.811692
Epoch 48
Validation binary_cross_entropy = 9.003356
Epoch 49
Loss = 1.4302e+01, PNorm = 53.3526, GNorm = 466.5849, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 4.770831
Epoch 50
Validation binary_cross_entropy = 4.712282
Epoch 51
Validation binary_cross_entropy = 8.983043
Epoch 52
Validation binary_cross_entropy = 9.667025
Epoch 53
Validation binary_cross_entropy = 3.954187
Epoch 54
Loss = 4.9821e+00, PNorm = 53.3925, GNorm = 279.3056, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 2.672851
Epoch 55
Validation binary_cross_entropy = 3.222256
Epoch 56
Validation binary_cross_entropy = 5.148495
Epoch 57
Validation binary_cross_entropy = 4.396750
Epoch 58
Validation binary_cross_entropy = 1.934487
Epoch 59
Loss = 4.8838e+00, PNorm = 53.4283, GNorm = 244.5174, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 1.738609
Epoch 60
Validation binary_cross_entropy = 3.079148
Epoch 61
Validation binary_cross_entropy = 3.180655
Epoch 62
Validation binary_cross_entropy = 1.738775
Epoch 63
Validation binary_cross_entropy = 1.887901
Epoch 64
Loss = 8.4789e+00, PNorm = 53.4614, GNorm = 173.8078, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 1.844560
Epoch 65
Validation binary_cross_entropy = 1.450092
Epoch 66
Validation binary_cross_entropy = 1.632313
Epoch 67
Validation binary_cross_entropy = 1.857720
Epoch 68
Validation binary_cross_entropy = 2.419052
Epoch 69
Loss = 5.0814e+00, PNorm = 53.4954, GNorm = 172.1355, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 3.105071
Epoch 70
Validation binary_cross_entropy = 1.379006
Epoch 71
Validation binary_cross_entropy = 1.114140
Epoch 72
Validation binary_cross_entropy = 2.885404
Epoch 73
Validation binary_cross_entropy = 1.008694
Epoch 74
Loss = 5.0246e+00, PNorm = 53.5266, GNorm = 218.5121, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 1.144774
Epoch 75
Validation binary_cross_entropy = 2.197990
Epoch 76
Validation binary_cross_entropy = 7.092903
Epoch 77
Validation binary_cross_entropy = 4.911701
Epoch 78
Validation binary_cross_entropy = 0.917991
Epoch 79
Loss = 4.4556e+00, PNorm = 53.5561, GNorm = 196.9470, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.968278
Epoch 80
Validation binary_cross_entropy = 1.087644
Epoch 81
Validation binary_cross_entropy = 1.867613
Epoch 82
Validation binary_cross_entropy = 4.745713
Epoch 83
Validation binary_cross_entropy = 2.456070
Epoch 84
Loss = 3.6998e+00, PNorm = 53.5814, GNorm = 176.6330, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.912743
Epoch 85
Validation binary_cross_entropy = 1.072792
Epoch 86
Validation binary_cross_entropy = 0.880792
Epoch 87
Validation binary_cross_entropy = 1.408008
Epoch 88
Validation binary_cross_entropy = 0.815693
Epoch 89
Loss = 2.5802e+00, PNorm = 53.6046, GNorm = 163.9040, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 1.074384
Epoch 90
Validation binary_cross_entropy = 0.901491
Epoch 91
Validation binary_cross_entropy = 0.908754
Epoch 92
Validation binary_cross_entropy = 0.963297
Epoch 93
Validation binary_cross_entropy = 0.658378
Epoch 94
Loss = 2.0504e+00, PNorm = 53.6265, GNorm = 203.1767, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.689851
Epoch 95
Validation binary_cross_entropy = 0.672191
Epoch 96
Validation binary_cross_entropy = 0.673340
Epoch 97
Validation binary_cross_entropy = 0.803446
Epoch 98
Validation binary_cross_entropy = 1.714782
Epoch 99
Loss = 4.0392e+00, PNorm = 53.6469, GNorm = 178.2973, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 1.772554
Epoch 100
Validation binary_cross_entropy = 0.660874
Epoch 101
Validation binary_cross_entropy = 0.515010
Epoch 102
Validation binary_cross_entropy = 0.557344
Epoch 103
Validation binary_cross_entropy = 0.493648
Epoch 104
Loss = 2.5213e+00, PNorm = 53.6659, GNorm = 103.4009, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.426076
Epoch 105
Validation binary_cross_entropy = 1.230773
Epoch 106
Validation binary_cross_entropy = 2.084329
Epoch 107
Validation binary_cross_entropy = 0.774988
Epoch 108
Validation binary_cross_entropy = 0.501006
Epoch 109
Loss = 2.8079e+00, PNorm = 53.6838, GNorm = 204.9706, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.647261
Epoch 110
Validation binary_cross_entropy = 0.581548
Epoch 111
Validation binary_cross_entropy = 0.533859
Epoch 112
Validation binary_cross_entropy = 1.052076
Epoch 113
Validation binary_cross_entropy = 1.091741
Epoch 114
Loss = 1.4166e+00, PNorm = 53.7006, GNorm = 98.6881, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.685799
Epoch 115
Validation binary_cross_entropy = 0.612827
Epoch 116
Validation binary_cross_entropy = 0.767464
Epoch 117
Validation binary_cross_entropy = 1.241620
Epoch 118
Validation binary_cross_entropy = 1.403131
Epoch 119
Loss = 1.9124e+00, PNorm = 53.7173, GNorm = 108.3623, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 1.042603
Epoch 120
Validation binary_cross_entropy = 0.737423
Epoch 121
Validation binary_cross_entropy = 0.583225
Epoch 122
Validation binary_cross_entropy = 0.484425
Epoch 123
Validation binary_cross_entropy = 0.516256
Epoch 124
Loss = 1.7475e+00, PNorm = 53.7330, GNorm = 114.4224, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.519306
Epoch 125
Validation binary_cross_entropy = 0.643681
Epoch 126
Validation binary_cross_entropy = 0.575007
Epoch 127
Validation binary_cross_entropy = 0.597277
Epoch 128
Validation binary_cross_entropy = 0.659311
Epoch 129
Loss = 1.3615e+00, PNorm = 53.7479, GNorm = 97.8804, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.752206
Epoch 130
Validation binary_cross_entropy = 0.850830
Epoch 131
Validation binary_cross_entropy = 0.646607
Epoch 132
Validation binary_cross_entropy = 0.392332
Epoch 133
Validation binary_cross_entropy = 0.377758
Epoch 134
Loss = 1.0111e+00, PNorm = 53.7616, GNorm = 80.6382, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.365137
Epoch 135
Validation binary_cross_entropy = 0.382361
Epoch 136
Validation binary_cross_entropy = 0.440236
Epoch 137
Validation binary_cross_entropy = 0.549292
Epoch 138
Validation binary_cross_entropy = 0.564149
Epoch 139
Loss = 1.0786e+00, PNorm = 53.7739, GNorm = 47.1810, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.499940
Epoch 140
Validation binary_cross_entropy = 0.438643
Epoch 141
Validation binary_cross_entropy = 0.426000
Epoch 142
Validation binary_cross_entropy = 0.443656
Epoch 143
Validation binary_cross_entropy = 0.558147
Epoch 144
Loss = 1.4663e+00, PNorm = 53.7847, GNorm = 108.7603, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.806966
Epoch 145
Validation binary_cross_entropy = 0.778247
Epoch 146
Validation binary_cross_entropy = 0.598469
Epoch 147
Validation binary_cross_entropy = 0.437779
Epoch 148
Validation binary_cross_entropy = 0.407211
Epoch 149
Loss = 1.1434e+00, PNorm = 53.7955, GNorm = 65.3832, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.401770
Epoch 150
Validation binary_cross_entropy = 0.423139
Epoch 151
Validation binary_cross_entropy = 0.439271
Epoch 152
Validation binary_cross_entropy = 0.460256
Epoch 153
Validation binary_cross_entropy = 0.506619
Epoch 154
Loss = 1.2910e+00, PNorm = 53.8064, GNorm = 64.3867, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.580883
Epoch 155
Validation binary_cross_entropy = 0.642715
Epoch 156
Validation binary_cross_entropy = 0.767525
Epoch 157
Validation binary_cross_entropy = 0.747432
Epoch 158
Validation binary_cross_entropy = 0.510986
Epoch 159
Loss = 1.0686e+00, PNorm = 53.8166, GNorm = 85.0207, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.399413
Epoch 160
Validation binary_cross_entropy = 0.447604
Epoch 161
Validation binary_cross_entropy = 0.718939
Epoch 162
Validation binary_cross_entropy = 0.746910
Epoch 163
Validation binary_cross_entropy = 0.629279
Epoch 164
Loss = 6.7870e-01, PNorm = 53.8263, GNorm = 88.4532, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.574482
Epoch 165
Validation binary_cross_entropy = 0.623021
Epoch 166
Validation binary_cross_entropy = 0.649986
Epoch 167
Validation binary_cross_entropy = 0.587423
Epoch 168
Validation binary_cross_entropy = 0.469700
Epoch 169
Loss = 8.6286e-01, PNorm = 53.8359, GNorm = 55.0423, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.428736
Epoch 170
Validation binary_cross_entropy = 0.449987
Epoch 171
Validation binary_cross_entropy = 0.427328
Epoch 172
Validation binary_cross_entropy = 0.422203
Epoch 173
Validation binary_cross_entropy = 0.459269
Epoch 174
Loss = 8.1953e-01, PNorm = 53.8454, GNorm = 86.8730, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.490914
Epoch 175
Validation binary_cross_entropy = 0.412109
Epoch 176
Validation binary_cross_entropy = 0.365918
Epoch 177
Validation binary_cross_entropy = 0.366106
Epoch 178
Validation binary_cross_entropy = 0.439126
Epoch 179
Loss = 1.1871e+00, PNorm = 53.8538, GNorm = 83.1550, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.521956
Epoch 180
Validation binary_cross_entropy = 0.542618
Epoch 181
Validation binary_cross_entropy = 0.480768
Epoch 182
Validation binary_cross_entropy = 0.432325
Epoch 183
Validation binary_cross_entropy = 0.394859
Epoch 184
Loss = 1.2328e+00, PNorm = 53.8609, GNorm = 58.1147, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.385009
Epoch 185
Validation binary_cross_entropy = 0.395058
Epoch 186
Validation binary_cross_entropy = 0.418260
Epoch 187
Validation binary_cross_entropy = 0.471515
Epoch 188
Validation binary_cross_entropy = 0.497934
Epoch 189
Loss = 5.4851e-01, PNorm = 53.8681, GNorm = 45.5978, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.531669
Epoch 190
Validation binary_cross_entropy = 0.574570
Epoch 191
Validation binary_cross_entropy = 0.516668
Epoch 192
Validation binary_cross_entropy = 0.444450
Epoch 193
Validation binary_cross_entropy = 0.419671
Epoch 194
Loss = 6.0680e-01, PNorm = 53.8750, GNorm = 44.7514, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.424335
Epoch 195
Validation binary_cross_entropy = 0.425228
Epoch 196
Validation binary_cross_entropy = 0.479911
Epoch 197
Validation binary_cross_entropy = 0.512097
Epoch 198
Validation binary_cross_entropy = 0.463997
Epoch 199
Loss = 8.6386e-01, PNorm = 53.8817, GNorm = 62.6641, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.409317
Epoch 200
Validation binary_cross_entropy = 0.357948
Epoch 201
Validation binary_cross_entropy = 0.354129
Epoch 202
Validation binary_cross_entropy = 0.374875
Epoch 203
Validation binary_cross_entropy = 0.457953
Epoch 204
Loss = 8.1554e-01, PNorm = 53.8882, GNorm = 67.0388, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.535827
Epoch 205
Validation binary_cross_entropy = 0.474225
Epoch 206
Validation binary_cross_entropy = 0.418721
Epoch 207
Validation binary_cross_entropy = 0.396867
Epoch 208
Validation binary_cross_entropy = 0.422593
Epoch 209
Loss = 6.1910e-01, PNorm = 53.8954, GNorm = 39.3349, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.460584
Epoch 210
Validation binary_cross_entropy = 0.505338
Epoch 211
Validation binary_cross_entropy = 0.536044
Epoch 212
Validation binary_cross_entropy = 0.530416
Epoch 213
Validation binary_cross_entropy = 0.481720
Epoch 214
Loss = 6.4739e-01, PNorm = 53.9016, GNorm = 36.1492, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.420164
Epoch 215
Validation binary_cross_entropy = 0.413126
Epoch 216
Validation binary_cross_entropy = 0.402540
Epoch 217
Validation binary_cross_entropy = 0.390644
Epoch 218
Validation binary_cross_entropy = 0.410912
Epoch 219
Loss = 6.5422e-01, PNorm = 53.9071, GNorm = 48.4023, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.430427
Epoch 220
Validation binary_cross_entropy = 0.431515
Epoch 221
Validation binary_cross_entropy = 0.420912
Epoch 222
Validation binary_cross_entropy = 0.409902
Epoch 223
Validation binary_cross_entropy = 0.416372
Epoch 224
Loss = 8.7440e-01, PNorm = 53.9129, GNorm = 89.8923, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.451923
Epoch 225
Validation binary_cross_entropy = 0.459825
Epoch 226
Validation binary_cross_entropy = 0.459313
Epoch 227
Validation binary_cross_entropy = 0.462119
Epoch 228
Validation binary_cross_entropy = 0.481016
Epoch 229
Loss = 5.8890e-01, PNorm = 53.9185, GNorm = 35.6052, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.492068
Epoch 230
Validation binary_cross_entropy = 0.524867
Epoch 231
Validation binary_cross_entropy = 0.549661
Epoch 232
Validation binary_cross_entropy = 0.571416
Epoch 233
Validation binary_cross_entropy = 0.594976
Epoch 234
Loss = 8.0734e-01, PNorm = 53.9235, GNorm = 54.9683, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.582395
Epoch 235
Validation binary_cross_entropy = 0.570472
Epoch 236
Validation binary_cross_entropy = 0.579860
Epoch 237
Validation binary_cross_entropy = 0.615422
Epoch 238
Validation binary_cross_entropy = 0.655661
Epoch 239
Loss = 7.0237e-01, PNorm = 53.9285, GNorm = 38.8283, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.673353
Epoch 240
Validation binary_cross_entropy = 0.659649
Epoch 241
Validation binary_cross_entropy = 0.618883
Epoch 242
Validation binary_cross_entropy = 0.572836
Epoch 243
Validation binary_cross_entropy = 0.510502
Epoch 244
Loss = 5.9794e-01, PNorm = 53.9330, GNorm = 49.0361, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.471034
Epoch 245
Validation binary_cross_entropy = 0.457062
Epoch 246
Validation binary_cross_entropy = 0.501226
Epoch 247
Validation binary_cross_entropy = 0.587184
Epoch 248
Validation binary_cross_entropy = 0.613481
Epoch 249
Loss = 6.7422e-01, PNorm = 53.9373, GNorm = 34.8958, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.556744
Epoch 250
Validation binary_cross_entropy = 0.520650
Epoch 251
Validation binary_cross_entropy = 0.496632
Epoch 252
Validation binary_cross_entropy = 0.479392
Epoch 253
Validation binary_cross_entropy = 0.466591
Epoch 254
Loss = 7.1367e-01, PNorm = 53.9415, GNorm = 69.3411, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.463325
Epoch 255
Validation binary_cross_entropy = 0.481370
Epoch 256
Validation binary_cross_entropy = 0.495944
Epoch 257
Validation binary_cross_entropy = 0.484831
Epoch 258
Validation binary_cross_entropy = 0.471357
Epoch 259
Loss = 3.6749e-01, PNorm = 53.9461, GNorm = 29.1159, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.454828
Epoch 260
Validation binary_cross_entropy = 0.438644
Epoch 261
Validation binary_cross_entropy = 0.461939
Epoch 262
Validation binary_cross_entropy = 0.541991
Epoch 263
Validation binary_cross_entropy = 0.567310
Epoch 264
Loss = 5.5106e-01, PNorm = 53.9505, GNorm = 42.8894, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.521124
Epoch 265
Validation binary_cross_entropy = 0.458945
Epoch 266
Validation binary_cross_entropy = 0.424233
Epoch 267
Validation binary_cross_entropy = 0.421913
Epoch 268
Validation binary_cross_entropy = 0.437556
Epoch 269
Loss = 7.7954e-01, PNorm = 53.9548, GNorm = 31.0571, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.459831
Epoch 270
Validation binary_cross_entropy = 0.460597
Epoch 271
Validation binary_cross_entropy = 0.455293
Epoch 272
Validation binary_cross_entropy = 0.436167
Epoch 273
Validation binary_cross_entropy = 0.417432
Epoch 274
Loss = 9.8714e-01, PNorm = 53.9589, GNorm = 78.4184, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.409731
Epoch 275
Validation binary_cross_entropy = 0.413996
Epoch 276
Validation binary_cross_entropy = 0.429454
Epoch 277
Validation binary_cross_entropy = 0.458177
Epoch 278
Validation binary_cross_entropy = 0.473344
Epoch 279
Loss = 7.5880e-01, PNorm = 53.9629, GNorm = 41.6790, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.466622
Epoch 280
Validation binary_cross_entropy = 0.470202
Epoch 281
Validation binary_cross_entropy = 0.485712
Epoch 282
Validation binary_cross_entropy = 0.503384
Epoch 283
Validation binary_cross_entropy = 0.523746
Epoch 284
Loss = 4.8465e-01, PNorm = 53.9668, GNorm = 23.7647, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.540264
Epoch 285
Validation binary_cross_entropy = 0.505658
Epoch 286
Validation binary_cross_entropy = 0.472494
Epoch 287
Validation binary_cross_entropy = 0.450273
Epoch 288
Validation binary_cross_entropy = 0.459333
Epoch 289
Loss = 6.1320e-01, PNorm = 53.9703, GNorm = 37.4442, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.475633
Epoch 290
Validation binary_cross_entropy = 0.502854
Epoch 291
Validation binary_cross_entropy = 0.524504
Epoch 292
Validation binary_cross_entropy = 0.489194
Epoch 293
Validation binary_cross_entropy = 0.428798
Epoch 294
Loss = 4.7257e-01, PNorm = 53.9741, GNorm = 28.2259, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.423025
Epoch 295
Validation binary_cross_entropy = 0.431072
Epoch 296
Validation binary_cross_entropy = 0.435922
Epoch 297
Validation binary_cross_entropy = 0.446696
Epoch 298
Validation binary_cross_entropy = 0.473261
Epoch 299
Loss = 5.2100e-01, PNorm = 53.9777, GNorm = 39.6385, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.451906
Model 0 best validation binary_cross_entropy = 0.354129 on epoch 201
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.382124
Ensemble test binary_cross_entropy = 0.382124
10-fold cross validation
	Seed 0 ==> test binary_cross_entropy = 0.588609
	Seed 1 ==> test binary_cross_entropy = 0.392099
	Seed 2 ==> test binary_cross_entropy = 0.428621
	Seed 3 ==> test binary_cross_entropy = 0.358662
	Seed 4 ==> test binary_cross_entropy = 0.387466
	Seed 5 ==> test binary_cross_entropy = 0.420375
	Seed 6 ==> test binary_cross_entropy = 0.713911
	Seed 7 ==> test binary_cross_entropy = 0.369915
	Seed 8 ==> test binary_cross_entropy = 0.422743
	Seed 9 ==> test binary_cross_entropy = 0.382124
Overall test binary_cross_entropy = 0.446453 +/- 0.108370
Elapsed time = 0:03:23
